Month: June 2021

Azure Synapse Link / Export to Data Lake Service – Performance (initial sync)


Recently we configured the Export to Data Lake service for one of our projects.

Just sharing the performance, we got during the initial sync.

Entity

Count

Contact

2,36,2581

Custom entity

1,61,3554

The sync started at 11:47 A.M and was completed around 4:50 P.M. – around 5 hours i.e. 300 minutes

Let us consider total records synced as 3975000 – 1613446 + 2362581 + 2 = 3976029.

So the performance here comes down to

  • 795000 records per hour
  • 13250 records per minutes
  • 220 records per second.

Of course, it will vary depending on the specific environment, table / entity type, attributes in it etc. it gives us a rough idea.

Posts on Azure Data Lake

Hope it helps..

Advertisements

Dataverse | Bypass Custom Business Logic


Great post on BypassCustomPluginExecution. If we are using KingswaySoft we can consider using Process Optimization options to disable relevant plugin execution – https://nishantrana.me/2020/11/12/process-optimization-dynamics-365-ce-crm-cds-ssis-integration-toolkit-kingswaysoft/

Rajeev Pentyala's avatarRajeev Pentyala – Technical Blog on Power Platform, Azure and AI

There are times when you want to be able to perform data operations without having custom business logic applied. Especially during complex data migration scenarios, we would not want our custom business logic (i.e., Sync Plug-ins/Real time Workflows) to be triggered.

The manual option to disable custom logic is,

  • We have to locate and disable the custom plug-ins. But this means that the logic will be disabled for all users while those plug-ins are disabled. It also means that you have to take care to only disable the right plug-ins and remember to re-enable them when you are done.

Now using BypassCustomPluginExecution option we can disable custom business logic programmatically.

About BypassCustomPluginExecution:

When you send requests that bypass custom business logic, all synchronous plug-ins and real-time workflows are disabled except:

  • Plug-ins which are part of the core Microsoft Dataverse system or part of a solution where Microsoft is the…

View original post 309 more words