Data migration: moving the meaning, not just the rows

Data migration is the part of every system change that decides whether the project succeeded, and it is the part most plans treat as an export button. Moving rows is trivial. The work is moving meaning: fields that were repurposed years ago, the same customer existing four times, history that a regulator can ask for, and totals that must tie out to the dollar on the other side. A migration done properly is a rehearsed, repeatable, reconciled process with a rollback — not a heroic weekend. Done that way, it is also the moment the organization finally cleans data it has been apologizing for since the old system was new.

Where this goes wrong:

The fields do not mean what their names say. Every long-lived system has a field that changed meaning in 2016, a status code that means one thing before a certain date and another after, and a notes column doing the work of five missing fields. A migration mapped from the schema instead of the data reproduces the labels and loses the meaning.

Identity is unresolved and the mess migrates with you. The same customer entered four times with three spellings does not become one customer by moving databases. If duplicate resolution is not an explicit step with explicit rules — what makes two records the same, which one wins each field — the new system inherits the old system's arguments on day one.

History gets dropped because it is inconvenient. Teams migrate open records and quietly abandon the closed ones, then discover the audit trail, the seven-year retention obligation, or the renewal conversation that depends on knowing what happened in 2021. What history moves, what gets archived read-only, and what is truly deleted is a decision to make deliberately — in Canada, with PIPEDA and Quebec's Law 25 in view — not a side effect of what the export handled.

Reconciliation by eyeball. Opening the new system, spot-checking ten records and declaring victory is how migrations pass on Friday and fail at month-end. If record counts, financial totals and sampled field-level comparisons are not computed on both sides and tied out, nobody actually knows the migration worked — they know it looks fine.

A one-way cutover with no rollback. The migration that runs once, manually, on the live weekend cannot be rehearsed and cannot be reversed. When something surfaces — and something surfaces — the choice is between limping forward on bad data and an unplanned restore that loses days of new work. Both are avoidable with a plan that assumed a second attempt.

How it actually gets built:

Profile the real data before promising anything. Before mapping a single field: null rates, value distributions, orphaned records, dates that predate the company, and the fields whose contents do not match their names. Profiling is a few days of work that converts the migration estimate from a guess into a plan, and it is where every unpleasant surprise is cheapest to find.

Map field by field, with an owner and a decision log. Every source field gets a destination, a transformation rule, or an explicit decision to leave it behind — and a named person who made that call. The mapping document is the contract for the whole migration; when a number looks wrong in March, the log says what was decided and why, instead of an archaeology project.

Build the migration as a repeatable script and rehearse it. The migration runs as code, end to end, against a copy — then again, and again, each rehearsal fixing what it surfaced and shrinking the runtime. By the real cutover the interesting question is not whether it works but how closely the numbers match the last rehearsal. Manual one-time migrations are rehearsal zero, run live.

Reconcile with numbers on both sides. Counts by record type, financial sums by period, and randomized record-level field comparisons, computed independently against source and destination and tied out. Discrepancies get explained, not waved through — a known, documented difference is acceptable; an unexplained one is a stop.

Cut over in stages, with the old system frozen and readable. The source system goes read-only at cutover so nothing diverges, and stays readable so any dispute can be checked against it. Where risk justifies it, a short parallel run on one workflow validates the new system against live reality before the rest follows. Rollback is a decision point on the plan, not an improvisation.

Archive the source and set an end date. The legacy database is preserved read-only with defined access and a defined retention period, and the decommission date is written down. Migrations that never formally end leave the organization paying for and half-trusting two systems — the exact condition the project existed to remove.

The AI question:

Migration is mapping, and mapping is now dramatically faster: AI reads a legacy schema and its actual contents and drafts the field mappings, flags the repurposed columns, proposes duplicate-matching rules and writes the transformation code — the labor that made migration quotes so heavy. What it does not do is own the outcome. Whether two similar records are one customer, which history the regulator can demand, and what a tied-out reconciliation must show are decisions someone accountable has to make, because the cost of getting them wrong lands months later, in production, on you.

VX-N uses AI across profiling, mapping, transformation and reconciliation tooling — one reason migrations that were quoted in months rehearse and land in weeks, with the numbers checked on both sides rather than assumed.

Our verdict: Migrate less than you think and more deliberately than the export button suggests. Open records and the history the operation actually consults move fully, cleaned and reconciled; deep history moves to a read-only archive with a retention decision attached; genuinely dead data is documented and left behind. Never accept a migration plan without profiling first, a rehearsal, or two-sided reconciliation — each absence is how the respective failure happens. And treat the migration as the one-time chance to fix identity and meaning: moving known-dirty data faithfully into a new system is paying for a fresh start and declining it.

How long does a legacy data migration take?

The honest driver is meaning, not volume — a million clean rows move faster than fifty thousand rows with repurposed fields and unresolved duplicates. After profiling, the estimate is real; rehearsals then compress the cutover itself to hours. Migrations quoted before anyone has looked at the actual data are quotes for a different dataset.

Does it matter whether the destination is Salesforce, SAP or a custom system?

Less than vendors suggest. The destination changes the loading mechanics and which objects exist to receive the data; the hard work — profiling, mapping, identity resolution, reconciliation — is identical. A team that has only ever loaded one platform tends to be strong on mechanics and silent on meaning.

How much history should we bring over?

Bring what the operation consults and what you are obliged to retain; archive the rest read-only. The right split is a business and compliance decision made per record type, not a technical default. Bringing everything slows the new system and imports old mess; bringing only open records quietly destroys the audit trail.

Our data is a mess. Should we clean it before or during migration?

During, as explicit transformation rules in the migration script — not before, by hand, in the old system. Scripted cleaning is rehearsed, repeatable and documented in the decision log; manual pre-cleaning is unrepeatable effort that drifts stale while the project runs, and nobody can say afterward exactly what was changed.

Can we migrate without downtime?

Close to it, with staging: history moves ahead of time, rehearsals compress the delta, and the final cutover moves only what changed since — typically a window of hours, scheduled when the operation is quiet. True zero-downtime synchronization is achievable but rarely worth its complexity outside round-the-clock operations.

Last reviewed 28 August 2026