Skip to main content

The Hopp Migration Approach

Hopp’s core functionality focuses on the mapping, transformation and validation of data for complex data migrations. This is critical to the delivery of a successful data migration but does not cover the full range of responsibilities of a data migration team within a wider implementation project. These pages will examine how additional Hopp optional functionality (the Planning Module and Item Manager) can be configured to support elements of a wider migration methodology.

Migration Methodology​

This is an overview of a wider methodology that can be used to ensure a smooth delivery of a data migration. It is based on Practical Data Migration by Johny Morris.

A data migration can be divided into two main phases:

  • Migration Preparation describes the preparatory tasks that should be carried out before starting to migrate data.
  • Migration Delivery outlines the processes to be followed when implementing the migration of data.

For each of these phases, this article will also consider the varying responsibilities of Hopp customers, depending on whether they are a System Integrator (SI) implementing a data migration to a new system, or a system supplier using Hopp to onboard new customers.

Migration Preparation​

In most cases, migration preparation will primarily be the responsibility of the organization that has decided to purchase a new system and migrate the data in their legacy data stores into it. In some cases, they may employ an SI to carry out much of this work on their behalf. Migration preparation can – and ideally should – begin before the new system has even been selected, which means that system suppliers are much less likely to be directly involved in most of these activities.

Migration preparation will begin with a set of initial requirements, which will be followed by a series of activities, as shown in the following diagram:

migration preparation diagram

It is not necessary to carry out the preparation activities in a set order. The diagram shows some relationships and dependencies between the activities, but it is often best to start wherever it is easiest.

Landscape Analysis​

This is often the largest single task during migration preparation. It is where relevant legacy data stores are identified and analyzed using modeling and profiling techniques.

It is unusual for system suppliers to undertake this activity, but organizations may commission a System Integrator to carry out this exercise on their behalf.

Legacy Data Store Register​

Legacy Data Stores discovered during Landscape Analysis should be catalogued in a register. The information held in this register will be an extremely valuable resource both for the data migration workstream itself and for the wider project, to support the benefit realization of systems that can be decommissioned after go-live.

Migration Issues Log​

Landscape analysis will start to identify issues that will need to be addressed by the data migration workstream. These will need to be recorded in a migration issues log. The migration issue management process is fundamentally central to the management of the entire data migration workstream. Therefore, it should remain independent of any other risk and issue logs, including the project risk and issue management and any IT fault management systems in place.

Key Data Stakeholder Register​

It is important that the data migration team knows who are the key people that they will need to work with to deliver a successful data migration. These will include:

  • Data Owners
  • Business Domain Experts
  • Technical System Experts
  • Data Architects
  • Data Management / MDM Experts

The list of these people should be maintained in a register. The initial requirements will probably identify some of these people, in particular the Data Owners for the main systems being replaced. Landscape Analysis should quickly start to identify others, especially the Business Domain Experts and Technical System Experts.

Establish Data Task Group​

The data migration workstream will need to establish a Data Task Group to manage the Migration Issues Log. This will consist of appropriate Key Data Stakeholders who have the delegated responsibility to be able to make decisions about how to deal with the issues logged.

The Data Task Group will be chaired by the lead migration analyst in the data migration team.

Where a System Integrator has been commissioned to deliver the data migration, this will be their senior migration analyst

The new system supplier’s data migration consultant will be a key data stakeholder with a place in the Data Task Group as a Technical System Expert

Project Decomposition​

This is necessary to organize the work of the data migration into manageable parts. This will commonly be organized along functional business area lines. It is often useful to ensure that each area has a single Data Owner (although the same Data Owner may be responsible for more than one area).

For a system supplier, it makes sense to ensure that the project decomposition aligns with the functional modules of their product.

Migration Sign-off Requirements​

When the scope is understood and Data Owners have been identified, migration sign-off requirements can be agreed with them. Data Owners will eventually be asked to sign off that they are ready for the new system to go live, so a good way to capture their requirements is to ask, “What would stop you from allowing your current systems to be turned off?” The answers to this question can then be turned into testing specifications.

System suppliers will have detailed knowledge of their system, so should be able to provide a comprehensive list of testing requirements. It is usually beneficial to bring a suggested list of requirements to the first meeting with a Data Owner.

Data Migration Strategy​

Everything learnt during migration preparation should feed into the data migration strategy. This will be a key project document that will also explain how the data migration workstream will relate to the wider project. This will include a data migration plan that is aligned with the wider project plan.

Migration Delivery​

The migration will be delivered during the implementation phase of the wider project to implement a new system. Initial timescales will be dictated by the project plan.

Data migration is an iterative process, with the following key elements:

  • Mapping
  • Testing
  • Issue Management
  • Sign-off

The following diagram shows how this fits together:

migration delivery diagram

Hopp’s core functionality supports the migration implementation indicated by the shaded box in the diagram above. The organization of this work will have been specified in the migration plan contained in the data migration strategy.

Migration Implementation – Milestones and Releases​

The wider project plan will have various milestones that the data migration will need to meet. The work of the data migration itself will be sub-divided into Releases that are analogous to Agile Sprints. Elements of the migration preparation will feed into the migration planning, in particular:

  • Project Decomposition will identify the different areas that need to be delivered by the data migration. This will help to define the focus for each Release.
  • Landscape Analysis will inform the data migration where the source data is and how it can be accessed.
  • The Migration Issues Log will provide information about the issues that will need to be addressed by the migration.

Testing​

As the data migration team begins to iteratively build the data migration, the results will need to be tested. In larger projects, the responsibility for testing is often held by a dedicated testing team, so communication of the specific migration testing requirements is vital.

There are different types of testing that are relevant to data migration, which include:

  • User Acceptance Testing(UAT): This is used to check that the new system operates as required. It is often initially carried out using test data, but it is important that real migrated data is also used.
    • This type of testing falls outside the direct scope of the core Hopp functionality.
  • Reconciliation Testing: This testing is used to confirm that the required records have migrated and check key values are correct. It often involves comparison of legacy system reports and outputs with their new system equivalents.
    • This type of testing can be supported by the core Hopp functionality but always requires external sources for comparison.
  • Record Level Testing: This type of testing is used to check individual records down to field level.
    • This type of testing is very well-supported by the core Hopp functionality.

Any errors identified in testing are logged as migration issues and fed back into the iterative migration loop.

Migration Issue Management​

The Data Task Group that was established during migration preparation will need to agree the priority and decide the plan for each issue raised. Where it is decided that action is required, the issue will be assigned to a Release. The Data Task Group will then be responsible for monitoring the progress of these issues.

Migration Issue Management is the central process that drives the Migration Delivery:

issue management process

  • Each new issue needs to be quantified to allow the Data Task Group to decide what needs to be done.
    • This is delivered by the Events from the core Hopp functionality.
  • The Data Task Group can then decide the priority and plan for the issue.
  • Issues that can be ignored are completed.
  • Other issues may need to be broken down into a series of sub-tasks.
  • These issues and any sub-tasks need to be managed.
  • When all tasks have been successfully carried out, the issue can be completed.
  • The progress of longer running issues will need to be monitored by the Data Task Group.

Migration Sign-off​

Before the new system can go live, the data migration will need to be signed off by the Data Owners. They will need to be provided with evidence that the migration sign-off requirements agreed during migration preparation have been satisfied. Production of this evidence should be the primary focus of the migration testing.