Load data from SQL On-Premise to CDS (Common Data Service) using Power Platform dataflows


Let us continue our previous post where we created a connection to the below On-Premise SQL Server Database using an On-premises data gateway.

https://nishantrana.me/2020/07/06/configuring-on-premises-data-gateway-to-connect-tosql-server-on-premise-data-source-power-platform/

Here we will use the Power Platform
dataflows to load contact entity in CDS using that same on-prem table.

Sign in to Power Apps

https://make.powerapps.com/home

Navigate to Data – Dataflows and create a new dataflow

Dataflow can store the entities either in Common Data Service or Azure Data Lake for analytical purposes.

Leave the Analytics entities only checkbox unchecked as we will be saving the entities in the Common Data Service.

Next, let us specify the data source. Here we have selected SQL Server Database as the source.

Specify connection details

On specifying the Server and Database details, it will auto-populate the rest of the connection details if already defined.

Select the table and click on Transform data

Transform Data give us the option of further refining the data to match the destination.

Click on Next and specify the mapping.

We can either load data in a new entity

Or load to an existing entity.

Here we have selected contact entity and clicked on Automap option which has mapped FirstName and LastName fields for us. Manually map the email field. Here we have not mapped the ContactID field for now.

Here for Key fields in Field Mapping, it will list down alternate key, if defined for the entity.

Specify the refresh settings. We have kept is as refresh manually.

With refresh completed, let us check the contact entity.

Go to Data – Entities and Data tab, we can see the records added.

Now let us add one more record in the source database and refresh the dataflow manually.

As we had not specified key while defining the data flow, we have all the records created again including the new one.

Now let us define an alternate key for the contact entity.

And edit the data flow to map the ContactID field of the source database to the newly created alternate key.

Define the mapping for the alternate key field.

Also, let us delete the old records that were synced. After refresh is done, we can see the records created

What about the update?

Let us update the record in source and run the refresh

As expected, the record is updated and no duplicate records are created in CDS.

What about delete?

Now let us delete the record in the source and run the refresh

The records are not deleted in the Common Data Service.

What about updates in CDS?

As this is a one-directional flow to load the data from source to destination, any change in CDS will be overridden and will not be synced back to the source SQL Server database on the next refresh run.

Let us update records in CRM, and run the refresh.

As expected, the record in CDS is updated back with source DB.

What about the history and log of the refresh?

Show refresh history option gives us all the details of the refresh run and the failure and success of the records.

More on the data flow capabilities

https://docs.microsoft.com/en-us/data-integration/dataflows/dataflows-integration-overview#dataflow-capabilities-in-power-platform-services

Also, check out the interesting article below

https://carinaclaesson.com/2020/05/11/alm-for-power-platform-dataflows-the-story-begins/

Hope it helps..

Advertisements

Author: Nishant Rana

I love working in and sharing everything about Microsoft.NET technology !

17 thoughts on “Load data from SQL On-Premise to CDS (Common Data Service) using Power Platform dataflows”

Please share your thoughts

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Power Melange

Power Melange By Shalinee

Clavin's Blog

Power Automate - Power Apps - SharePoint Online - Azure - Nintex - K2 - Artificial Intelligence

Sat Sangha Salon

An Inquiry in Being

The Indoencers

The Influencers & Influences of Indian Music

Monika Halan's blog

Hand's-free money management

D365 Demystified

A closer look at Microsoft Dynamics 365.

Microsoft Mate (msftmate) - Andrew Rogers

Experienced consultant primarily focused on Microsoft Dynamics 365 and the Power Platform

Knowhere365

Specific topics by Django Lohn on the whole Microsoft365 Stack

Manmit Rahevar's Blog

One Stop Destination for Microsoft Technology Solutions

MG

Naturally Curious

Brian Illand

Power Platform and Dynamics 365

Steve Mordue MVP

A Microsoft Business Applications MVP

Subwoofer 101

Bass defines your home theater

SQLTwins by Nakul Vachhrajani

SQL Server tips and experiences dedicated to my twin daughters.

Everything D365

Discovering Azure DevOps and D365 Business Applications

Tech Wizard

Lets do IT Spells

Two Bite Tips

Valuable Tips To Grow Your Business

XRM Tricks (Power Platform & Dynamics CRM )

Power Platform & Dynamics CRM

CRM TIPS BY PRM

Mail to crmtipsbyprm@gmail.com for queries and suggestions

nijos.dev

Giving back to the community what I have learned

xrm CRM Dynamics

Dynamics CRM Technical & Functional Info

Dynamics 365 Blogs - Explained in unique way

Sometimes you need to look at things from different perspective.

CRM Keeper

Dynamics 365 Customer Engagement, CRM, Microsoft CRM, Dynamics CRM

EVOLVED365

Step into the world of a Dynamics 365 Consultant

Dianamics PCF Lady

Diana & Dynamics 365 & Power Platform

Sara Lagerquist

No Code Customization Concepts

innovativeaj

developer, designer, writer, fun loving, patriotic, humble and a sweet person inside out :) love and respect people who have a great sense of humor.

Temmy Wahyu Raharjo

Dreaming to be a clean coder and TDD minded programmer.

Transform 365

We blog about problems we face and code we write to help others

Virendra Agrawal's Blog

Blogs, News and Insights of Dynamics 365 world

Jukka Niiranen

Thinking forward about Microsoft Power Platform

Amar Singh

Share your knowledge. It’s a way to achieve immortality.

%d bloggers like this: