Insights

Choosing the right data migration strategy

Author: Eric Valenzuela

Migrating to the cloud offers multiple benefits. Organizations looking to modernize can reap enormous advantages that range from decreased licensing and storage costs to increased performance and scalability. Migrating to the cloud enables organizations to advance their data analytics and even drive new revenue streams.

Of course, detailed planning and preparation are a must if you want to increase the success of your data migration. And choosing the right data migration strategy is a key part of this process.

I’ve participated in countless data migrations. Below, I’ll overview the four approaches that I most often see, as well as their advantages and disadvantages. For additional help choosing the right strategy, be sure to check into a data migration assessment.

1. Lift and shift approach

The “lift and shift” is the most basic data migration strategy. Organizations use this approach when taking the same database technology that they’re using on premise—say, an Oracle database—and shifting it to the cloud.

The main advantage of this approach is its simplicity and speed and the availability of tools that can facilitate the migration. But keep in mind that when you lift and shift an existing database, you’re also moving all of its hidden problems, hacks, and band-aids to the cloud. The technical debt you had on premise will still be an issue in the cloud—sort of like moving out of a roach-infested house and taking the roaches with you to your new home.

2. Data lake approach

The data lake approach involves removing data from an on-premise database and storing it in a solution such as Amazon S3. For this approach, you can employ a variety of available technologies to access, analyze, and query the data. This approach also gives you the ability to load the data into your database of choice, such as Redshift. Plus, because there’s a middle ground where the data sits, you can load it into the target database in parallel.

This approach offers the flexibility to use any database. While it does require redesigning your database to some degree, you can make changes to the data along the way and increase the performance in your target database. Additionally, because you are building something new and modernizing your database application, you can exterminate those roaches.

Additionally, this approach enables you to separate storage from compute, so you can take advantage of newer compute technologies as they emerge. There are also cost advantages of separating data that is accessed seasonally from data that is accessed regularly. A data lake allows you use compute power to handle those spikes and shut if off when the data isn’t being used.

Of course, there are some tradeoffs. Direct queries against a data lake are typically slower than queries against a database. In situations where you have a higher performance requirement, it makes sense to load that data into a higher performing database. But you can still load your data from the data lake into a tool that allows that high performance. Another tradeoff is that data lakes introduce a new set of technologies to engineers and DBAs who must contend with a steep learning curve. This new domain of knowledge requires considerable ramp up.

3. Combo approach: Lift and shift + data lake

Sometimes, there is a pressing business need to migrate—such as a datacenter that is shutting down. The timeline is short, and the organization needs to get the database off premise quickly—before migrating to a new database solution.

In this hybrid approach, you can combine the above-mentioned strategies, first using the lift and shift approach to migrate the existing database to the cloud, then, taking the time to migrate to a new database such as Redshift or Aurora.

In addition to getting to the cloud quickly, this approach gives you breathing room to determine the best database to be used for your data. The disadvantage is that in some cases, it ends up creating more work in the long run, since you have to make your database work in the cloud. And then you have to migrate to a new database technology.

4. Hybrid streaming approach 

In some cases, an organization wants the benefits of a data lake in the cloud, but for business reasons must keep an existing transactional database on premise. Using change data capture (CDC), you can enable changes on premise to stream into the cloud in near real-time. By taking a streaming approach to getting data into the cloud, you can get benefits from the cloud without moving the database. On the downside, replication from the on-premise database is more prone to failure since typically the same redundancies aren’t in place.

Choose the right strategy with a data migration assessment

Choosing the right data migration strategy is critical to your overall success, and depends upon business requirements and drivers, pain points, performance needs, short-term plans, future goals, and multiple other factors.

A data migration assessment from Beyondsoft applies a holistic approach to preparing for a data migration and provides a sound foundation and blueprint for completing your database and application migration. Our proven execution model addresses areas such as business applications and workloads, technical operations, production support, and training to determine the best migration architecture and strategy. Our experts work with you to perform a detailed analysis of your existing database and application architecture and create a roadmap for your migration that includes business, functional, technical, and testing requirements as well as recommendations for addressing gaps. We also create an ROI model that demonstrates performance measures and the time it will take to recoup the cost of your migration.

Here at Beyondsoft, we’ve performed countless database migrations and big data projects for large enterprise customers around the globe leveraging technologies such as PostgreSQL, Amazon RDS, Amazon Redshift, Vertica, Presto, ConvergDB, and more. A data migration assessment is a worthwhile investment that will save you time and money down the road. To learn more, talk to one of our data migration experts.