Record count tells you almost nothing about migration effort. Data quality tells you everything. This is why migration estimates are wrong so consistently — they are made before anyone has looked.
Moving 40,000 clean records is a day’s work. Moving 40,000 records where the same customer appears four times under three spellings, dates sit in two formats and a third of phone numbers live in a notes field is several weeks.
Why it is underestimated
It is quoted from a conversation. “How many customers do you have?” “About 40,000.” “Fine, two weeks.” Nobody opened the file.
Nobody owns the data. It accumulated over a decade across several people and a system change. No one person knows what is in it.
The problems are invisible until you try. Everything looks fine in a spreadsheet view. The duplicates, the inconsistent formats and the orphaned records surface when you attempt a structured import.
It is bundled into a fixed price. Which means the supplier guessed, and guessing wrong on migration is one of the main routes to a 50% overrun.
This connects directly to the broader pattern: the Panorama 2026 survey found 54.9% of over-budget ERP projects needed technology discovered after the start and 51% saw scope grow. Migration is where both frequently originate.
What actually makes it hard
Duplicates that are not exact
J Smith Ltd, J. Smith Limited, Smith (J) Ltd and JSMITH LTD are one customer. No automated match catches all of these, and merging requires judgement about which record’s data is correct.
Fields used for more than one thing
A notes field containing delivery instructions, phone numbers, account warnings and somebody’s reminder about a 2019 dispute. All of it matters to someone; none of it is structured.
Records that reference things that no longer exist
Orders pointing at deleted products. Invoices against customers who were merged. Stock movements referencing a warehouse that closed. The old system tolerated it; the new one enforces referential integrity and rejects them.
Historical data that was never right
Stock counts that have not matched reality since an incident two years ago. Opening balances nobody can reconcile. Migration is often the moment a business discovers its data has been wrong for years — which is valuable and is not in anyone’s plan.
Multiple formats in one column
Dates as 01/02/2024, 2024-02-01, 1 Feb 24 and Feb 2024 in the same field, because three people entered them over six years.
Encoding and character issues
Accented characters, currency symbols and apostrophes that became question marks in a previous migration. They look like one character problem and affect several thousand records.
The six-stage process
1. Inspect before quoting
Non-negotiable. A real extract, examined, before any number is given.
What gets checked: record counts per entity, duplicate rate, null rates on required fields, format consistency, referential integrity, encoding.
Output: a data quality report and a migration quote based on evidence rather than optimism.
This takes one to three days and it is the difference between an accurate quote and a guess.
2. Decide what moves
Not everything should. Common and sensible decisions:
- Customers and suppliers: all active, plus inactive with history
- Transactions: typically 2–7 years, with older data archived rather than migrated
- Stock: current positions, not full movement history
- Attachments: often the largest volume and the lowest value — decide deliberately
Archive what you do not migrate into something readable. You rarely need it, and when you do, you need it badly.
3. Clean at source where possible
Cleaning in the old system is usually better than cleaning in transit, because the people who understand the data are the people using it daily, and the cleanup has immediate value even if the project is cancelled.
Give the team a duplicate list and a bad-format report and a few weeks. They will fix more than a script can, and correctly.
4. Build the mapping
Field by field: source, destination, transformation rule, default where empty, and what happens when the rule fails.
Every exception is a decision someone must make. Expect a list of questions — that is the process working.
5. Rehearse, repeatedly
Three full rehearsals minimum, each against a fresh extract, each producing reconciliation reports.
| Reconciliation check | What it proves |
|---|---|
| Record counts in vs out, by entity | Nothing silently dropped |
| Financial totals: balances, debtors, creditors | The one finance will check |
| Stock quantity and value totals | Inventory integrity |
| Rejected record log with reasons | Nothing failed invisibly |
| Spot-check sample, verified by a human | Fields landed where intended |
Rehearsal three should produce no surprises. If it does, you are not ready to go live.
6. Cut over with parallel running
Final migration, then run both systems until the numbers reconcile. Do not decommission anything on faith.
Cleaning data without a supplier
Work your team can start today, which reduces the migration quote and has value even if no project follows.
Week 1 — see the problem. Export customers, suppliers and products to a spreadsheet. Sort by name. Scroll. You will find the near-duplicates within minutes, and the exercise changes most people’s estimate of the work.
Week 2 — count it. How many records are duplicates, roughly? What percentage of a required field is empty? How many different date formats appear in one column? Three numbers, and they turn a migration guess into an estimate.
Weeks 3–6 — fix the worst. Give the duplicate list to the people who know the customers. They will merge correctly because they know which address is current. A script will merge confidently and wrongly.
Ongoing — stop it regenerating. Agree entry standards and add validation where the system allows. A cleanup without a standard is a cleanup you repeat in two years.
What not to do: attempt a perfect cleanse before starting the project. Perfect is not required and chasing it delays everything. Fix the duplicates and the format inconsistencies in the fields that matter, and let migration handle the long tail with a rejection log.
What it costs
| Situation | Typical cost |
|---|---|
| Clean data, modern source, documented structure | £5,000 – £10,000 |
| Typical SME data, moderate quality issues | £10,000 – £20,000 |
| Poor quality, multiple sources, significant cleanup | £20,000 – £40,000 |
| Legacy system with no export, undocumented structure | £30,000+ |
Insist it is quoted separately, after inspection. A migration bundled invisibly into a project price is a number someone guessed, and you will meet the consequences at the worst moment.
What goes wrong on cutover day
The failures that actually occur, and what prevents each.
The reconciliation does not balance. Finance cannot sign off and nobody knows whether the difference matters. Prevented by: three rehearsals with the same reconciliation reports, so the first time finance sees them is not on the day.
A source system had data nobody knew about. A spreadsheet one department maintained separately, discovered when they ask where their records went. Prevented by: a source inventory built from interviews with the people doing the work, not from the system list.
The extract taken on the day differs from the rehearsal extract. New records, new edge cases, a field someone started using last month. Prevented by: rehearsing against fresh extracts each time, not the same frozen copy.
Attachments did not come across. Usually the largest volume and the lowest priority, so it slips. Prevented by: deciding explicitly whether attachments migrate, and testing it if they do.
Nobody agreed the point of no return. Half-migrated, something is wrong, and there is no decision about whether to continue or roll back. Prevented by: a written rollback plan with a named decision-maker and a time by which the call is made.
What to insist on
- Migration quoted separately, after a real extract has been inspected
- A written data quality report before the quote
- A documented field mapping you can review
- At least three full rehearsals
- Reconciliation reports your finance team signs off
- A rejected-record log with reasons, reviewed each time
- Parallel running until the numbers agree
- The migration scripts handed over to you at the end
That last point matters more than it appears. Migration scripts are part of the deliverable — you may need them again, and they document what happened to your data.
What your team must do
Migration is not something a supplier does to you. It needs people who understand the data.
Someone who knows the history. Why there are two customer records for the same company. What the notes field means.
Someone from finance to verify totals. They are the only people who will spot a balance that is subtly wrong.
Someone to make the decisions. Which duplicate is correct, how far back to migrate, what gets archived. Expect dozens of these questions.
Time allocated, not squeezed in. Migration questions blocking for a week is a week of project delay.
A migration plan that works
The document your supplier should produce before migration starts. If one does not exist, that is the finding.
1. Scope. Which entities move, which are archived, how much history. Agreed in writing, because “all of it” is a decision nobody has actually made.
2. Source inventory. Every system data comes from, including the spreadsheets. There are always more sources than the project assumes.
3. Field mapping. Source field, destination field, transformation rule, default where empty, and what happens when the rule fails. The last column is the one people omit and the one that generates the decision list.
4. Cleansing plan. What gets fixed in the source, what gets fixed in transit, who does each, and by when.
5. Rehearsal schedule. Dates, each against a fresh extract, with reconciliation output reviewed after each.
6. Reconciliation specification. Which totals must match, to what tolerance, and who signs them off. Finance signs the financial ones.
7. Cutover plan. The sequence on the day, who does what, how long each step takes, and the point of no return.
8. Rollback plan. What happens if the reconciliation fails on cutover day. A migration without one is a migration that cannot be stopped.
The decisions only you can make
Migration generates dozens of these. Expect them, and allocate someone’s time to answering them, because each one waiting a week is a week of project delay.
Which duplicate is correct? Where the same customer exists four times, someone must decide which record’s address, terms and contacts survive. A script cannot.
How far back? Transactions, attachments, correspondence. Every extra year costs money and most of it is never read again.
What about the orphans? Orders referencing deleted products, invoices against merged customers. The old system tolerated them; the new one will reject them. They get fixed, excluded or archived — and that is a business decision.
What do the free-text fields mean? The notes field containing delivery instructions, phone numbers and a reminder about a 2019 dispute. Someone must decide what gets structured and what gets discarded.
What is the opening position? Stock counts and balances at cutover. Finance owns this and it should not be a surprise on the day.
Who signs off? One named person per entity type, with authority to say the migration is acceptable.
Archiving what you do not migrate
The decision people defer and then regret, usually at the worst moment.
You will not migrate everything, and you should not. Ten years of transactions, superseded product records, correspondence from suppliers you no longer use — moving it costs money, slows the migration and clutters the new system.
But you will need some of it. An HMRC enquiry, a customer dispute, a warranty claim, a legal matter. Record retention obligations in the UK typically run six years for accounting records, and longer in some sectors.
What a usable archive looks like:
- Readable without the old system. A database backup you can only open with software you no longer licence is not an archive.
- Structured exports — CSV or similar, one file per entity, with a documented schema.
- Attachments extracted, with a reference linking each to its record.
- A plain-English index saying what is in there, what period it covers, and how to find something.
- Stored somewhere durable and tested — restore a sample before you decommission anything.
Do this before switching the old system off, not after. Once the licence lapses or the server is decommissioned, extraction becomes a specialist job and sometimes an impossible one.
Frequently asked questions
How much does ERP data migration cost?
£5,000–£10,000 for clean data from a modern system; £10,000–£20,000 for typical SME data; £20,000–£40,000 where quality is poor or there are multiple sources. It should be quoted separately after a real extract has been inspected.
Why does data migration always take longer than planned?
Because it is estimated before anyone looks at the data. Record count is irrelevant; quality is everything. Duplicates, inconsistent formats, orphaned records and multi-purpose fields are invisible until you attempt a structured import.
How long does ERP data migration take?
Two to four weeks for clean data; six to twelve weeks where there is significant cleanup, running alongside the build rather than after it.
Should I clean my data before migration?
Yes, in the source system where possible. Your team understands the data better than any script, and the cleanup has value even if the project never completes.
How much history should I migrate?
Typically two to seven years of transactions, all active customers and suppliers, current stock positions. Archive the rest somewhere readable rather than migrating it.
What is a reconciliation report?
A comparison proving the migration was complete and correct — record counts in and out, financial totals, stock totals, and a log of rejected records with reasons. Your finance team should sign it off before go-live.
How many rehearsals do we need?
At least three, each against a fresh extract. The third should produce no surprises. If it does, you are not ready.
Written by Jas Singh, Founder. Last updated 8 October 2026.
Overrun cause figures from the Panorama Consulting 2026 ERP report (170 organisations, January 2025 – January 2026) as reported by secondary sources; cited as directional.
