How to – Optimize Delete operation – Dataverse / Dynamics 365


Recently we had to delete records for one of our entities, and we tried out different combinations of batch sizes and the number of threads with 25000 records as a sample to find the optimum setting.

Below is our sample SSIS Package (uses KingswaySoft Dynamics 365 Tool Kit), it retrieves 25000 record’s GUID (Contact table / entity) and then distributes it equally among 3 different CRM Destination Component running under different users (CRM Connection Managers).

How to – improve data migration performance – SSIS & Azure Data Factory (Dataverse / Dynamics 365)

Below is our Premium Derived Column where we have added a new column with expression IncrementValue()

In Conditional Split component, we are then using this new column added to distrubute the output across three CRM Destination Component, each using a different CRM Connection Managers running under different application users.

We first started with 10 batch size and 20 thread followed by different combinations after that à

Below were our findings ->

Records Count Batch Size Thread Parallel Users Elapsed Time
25000 10 20 3 00:15:58.578
25000 10 15 3 00:14:43.734
25000 10 10 3 00:16:06.438
25000 10 5 3 00:23:52.094
25000 10 15 2 00:18.55.012
25000 10 15 1 00:39:15.828
25000 20 30 1 00:39:12.781

As we can see the Batch size 10 and thread around 15 gave us the best performance. However, evert environment / conditions will would be different so we should try out different combinations before finalizing.

SSIS and Microsoft Dynamics 365

Hope it helps..

Advertisements

Author: Nishant Rana

I love working in and sharing everything about Microsoft.NET technology !

3 thoughts on “How to – Optimize Delete operation – Dataverse / Dynamics 365”

Please share your thoughts

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Welcome to the Blog & Website of Paul Andrew

Technical Leadership Centred Around the Microsoft Data Platform

Deriving Dynamics 365

Deriving Solutions and features on Power Platform/Dynamics 365

The CRM Ninja

Thoughts & musings from a Dynamics 365 Ninja!

CRM for You and Me

Customer Relationship Mgt App for the Professional

D CRM Explorer

Learn about Microsoft Dynamics CRM Power Platform customization and implementation and other cool stuffs

Stroke // Jonas Rapp

I know pre-stroke. I will improve who I was.

Power Melange

Power Melange By Shalinee

Clavin's Blog

Power Automate - Power Apps - SharePoint Online - Azure - Nintex - K2 - Artificial Intelligence

Sat Sangha Salon

An Inquiry in Being

The Indoencers

The Influencers & Influences of Indian Music

Monika Halan's blog

Hand's-free money management

D365 Demystified

A closer look at Microsoft Dynamics 365.

Microsoft Mate (msftmate) - Andrew Rogers

Experienced consultant primarily focused on Microsoft Dynamics 365 and the Power Platform

Knowhere365

Specific topics by Django Lohn on the whole Microsoft365 Stack

Manmit Rahevar's Blog

One Stop Destination for Microsoft Technology Solutions

MG

Naturally Curious

Brian Illand

Power Platform and Dynamics 365

Steve Mordue MVP

A Microsoft Business Applications MVP

Subwoofer 101

Bass defines your home theater

SQLTwins by Nakul Vachhrajani

SQL Server tips and experiences dedicated to my twin daughters.

Everything D365

Discovering Azure DevOps and D365 Business Applications

Tech Wizard

Lets do IT Spells

XRM Tricks (Power Platform & Dynamics CRM )

Power Platform & Dynamics CRM

CRM TIPS BY PRM

Mail to crmtipsbyprm@gmail.com for queries and suggestions

nijos.dev

Giving back to the community what I have learned

xrm CRM Dynamics

Dynamics CRM Technical & Functional Info

Dynamics 365 Blogs - Explained in unique way

Sometimes you need to look at things from different perspective.

CRM Keeper

Dynamics 365 Customer Engagement, CRM, Microsoft CRM, Dynamics CRM

EVOLVED365

Step into the world of a Dynamics 365 Consultant

Dianamics PCF Lady

Diana & Dynamics 365 & Power Platform

Sara Lagerquist

No Code Customization Concepts

innovativeaj

developer, designer, writer, fun loving, patriotic, humble and a sweet person inside out :) love and respect people who have a great sense of humor.

Temmy Wahyu Raharjo

dreaming to be a clean coder and TDD minded programmer.

%d bloggers like this: