Data Migration in Dynamics 365 – Key considerations


I was recently watching the FastTrack TechTalks on Data Migration.

https://community.dynamics.com/365/b/techtalks/posts/data-migration-january-16-2018

Below are some of the key points from the session à

Customer 360 View is more often than not, the starting point of the data migration discussions. Storing all the data in Dynamics 365 might not be a good idea as it could impact the performance and more importantly, the storage has a cost associated with it.

Learn more about the storage in Power Platform here

https://docs.microsoft.com/en-us/power-platform/admin/whats-new-storage

Important Considerations as part of the Data Migration Strategy

Data Source

  • What is the source of the migration? Is it a single system or multiple systems from which the data needs to be consolidated.
  • The appropriate connectors or technology needed to fetch or extract the data from the source system(s). (SQL, File-based, Web Service, etc.)
  • Need for a staging database for transformation and cleansing of data before migrating it to Dynamics 365.
  • Having a well-defined Dynamics 365 data dictionary and model.

Data Cleansing

  • Removal of the old data.
  • De-duplicate of the data.
  • Checking for null, missing required fields and references.
  • Identify and define the uniqueness of the record.
  • Need for defining Alternate keys

Data Transformation (mapping with Dynamics 365 entities and fields)

  • Transform the data to adapt to the target Dynamics 365 Data model.
  • Format and Range of the fields.
  • Calculate or combination of different fields.
  • Filtering and Enriching of the fields

Mapping OptionSet in KingswaySoft SSIS Integration Toolkit for Dynamics 365

Mapping OptionSet in Scribe Online

Lookup and Cache Transformation component in KingswaySoft SSIS Integration Toolkitfor Dynamics 365

Data Loading

  • Is it one-time migration (plus delta load daily) or n times migration
  • Based on the time it takes, the batch can run over the weekend or after office hours daily.
  • Order of data load – Master \ Configuration Data à Parent records àChild records.
  • Use Create or Update whenever possible instead of Upsert.
  • For Inactive records, first, we need to create those records as active, associate any child records to it if any, before updating the status of the record.
  • Use multithreading and execute multiple requests.
  • Figure out the optimum batch size and thread.

Optimum batch size

For Upsert in KingswaySoft SSIS Integration Toolkit for Dynamics 365

For Delete in KingswaySoft SSIS Integration Toolkit for Dynamics 365

Volume

How many records are to be migrated and how much time will it take.

Storage / Scalability

Do we need all the data to be stored in Dynamics 365, or for Customer 365, can we leverage Power BI reports which could pull data from different data sources?

Do we need all the data for all the entities or last x months?

Compliance and regulations

We need to consider what information is allowed to be stored/accessed by the users in Dynamics 365.

Other key points

  • Use WebAPI for migration.
  • Use Execute Multiple for batch operations.
  • Leverage Application User that provides Server to Server authentication.
  • To update CreatedOn field use OverriddenCreatedOn field.
  • Use Create OnBehalfOf / CallerID for Created By field.
  • Map ModifiedBy and ModifiedOn to custom fields for the historical purpose.
  • Check for the sequence of the auto-number fields in the entities.
  • If possible for improved performance – during migration disable duplicate detection, workflows, plugins, audit, activity feeds, etc.
  • Leverage the same location/region for running the package as the Dynamics 365.
  • For faster date-time conversions set regional settings to UTC for the service account being used.

Check out the below articles

https://kunaltripathy.com/2019/06/04/dynamics-365-data-migration-demystified-part-i/

Hope it helps..

Author: Nishant Rana

I love working in and sharing everything about Microsoft.NET technology !

2 thoughts on “Data Migration in Dynamics 365 – Key considerations”

Please share your thoughts

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

CRM for You and Me

Customer Relationship Mgt App for the Professional

D CRM Explorer

Learn about Microsoft Dynamics CRM Power Platform customization and implementation and other cool stuffs

Stroke // Jonas Rapp

I know pre-stroke. I will improve who I was.

Power Melange

Power Melange By Shalinee

Clavin's Blog

Power Automate - Power Apps - SharePoint Online - Azure - Nintex - K2 - Artificial Intelligence

Sat Sangha Salon

An Inquiry in Being

The Indoencers

The Influencers & Influences of Indian Music

Monika Halan's blog

Hand's-free money management

D365 Demystified

A closer look at Microsoft Dynamics 365.

Microsoft Mate (msftmate) - Andrew Rogers

Experienced consultant primarily focused on Microsoft Dynamics 365 and the Power Platform

Knowhere365

Specific topics by Django Lohn on the whole Microsoft365 Stack

Manmit Rahevar's Blog

One Stop Destination for Microsoft Technology Solutions

MG

Naturally Curious

Brian Illand

Power Platform and Dynamics 365

Steve Mordue MVP

A Microsoft Business Applications MVP

Subwoofer 101

Bass defines your home theater

SQLTwins by Nakul Vachhrajani

SQL Server tips and experiences dedicated to my twin daughters.

Everything D365

Discovering Azure DevOps and D365 Business Applications

Tech Wizard

Lets do IT Spells

Two Bite Tips

Valuable Tips To Grow Your Business

XRM Tricks (Power Platform & Dynamics CRM )

Power Platform & Dynamics CRM

CRM TIPS BY PRM

Mail to crmtipsbyprm@gmail.com for queries and suggestions

nijos.dev

Giving back to the community what I have learned

xrm CRM Dynamics

Dynamics CRM Technical & Functional Info

Dynamics 365 Blogs - Explained in unique way

Sometimes you need to look at things from different perspective.

CRM Keeper

Dynamics 365 Customer Engagement, CRM, Microsoft CRM, Dynamics CRM

EVOLVED365

Step into the world of a Dynamics 365 Consultant

Dianamics PCF Lady

Diana & Dynamics 365 & Power Platform

Sara Lagerquist

No Code Customization Concepts

innovativeaj

developer, designer, writer, fun loving, patriotic, humble and a sweet person inside out :) love and respect people who have a great sense of humor.

Temmy Wahyu Raharjo

Dreaming to be a clean coder and TDD minded programmer.

Transform 365

We blog about problems we face and code we write to help others

%d bloggers like this: