Data migration is fun!

Data Migration 

Data migration is not the fun bit.  When you’re upgrading your business’ technology stack with a new system the new app is the fun bit, the automation, the chatbot… that stuff.  But not the data migration.  

Why isn’t it fun? 

There’s a few awkward bits: firstly, before you can kick off data migration you need to get hold of all your data. As a first step this involves letting your old girlfriend (software house) know you’ve got a new one.  Then, as they’re processing that little bombshell, they’re asked for a machine-readable data dump for a transfer.  Like asking a newly ex-ified girlfriend if she wouldn’t mind just transcribing all your years of conversations and providing them in a4 notebooks with equally spaced columns, grouped by date.   This has to be tactfully managed to avoid being a real dump of a data dump.  

How it gets into the new system

There are a number of ways to get data into a new system.  Not all of them are horrible, and there are often lots of benefits to the process.  The basic options (all of which Ignite has done as some point) are: feeding the data through a validated API configured to requirements; using the automation framework to input risk details through the UI’s validation; rekey.  Horses for courses, and none are options to be ruled out.  

Benefits of data migration

Lots of companies have pre-GDPR data that they haven’t dealt with appropriately.  Data transfer is an excellent way of correcting this issue.  Data can also get a bit mangled over time with inconsistent inputs or database types.  Data transfer, especially through a validated API or automation framework offers the opportunity to cleanse data into formats that can better reported on and understood.  

Business risk

Data transfer is perceived as a business risk.  It is true to an extent, but these days it is increasingly seamless: test runs can be done in minutes, and mapping is increasingly easy.  Part of the skill comes in designing the new system.  If, in migrating, the existing product structure is largely retained, with upgrades to the functionality around it rather than the data capture itself, then migration can be easier.  Also consider how much data really needs to be transferred, and what can simply sit in an external data repository to be used for uncommon events like an investigation.  

Cost

Data transfer no longer needs to be expensive.  As long as there is decent access to historic data in some sort of machine readable format (e.g. historic EDI or Bordereaux) then it is something that, with proper management, can be done without headache or heartache.