Category: Microsoft Dynamics 365

5000 records limit of Fetch XML Query – List records action of Common Data Service (current environment) connector in Power Automate


Recently in one of our requirements, while trying to fetch contact records using Fetch XML Query

https://docs.microsoft.com/en-us/connectors/commondataserviceforapps/#list-records

we realized that we can only retrieve / fetch 5000 records.

Enabling pagination will also not help here (the Next link also come as null).

The alternate solution is to convert the Fetch XML query to OData and use the same inside List records action

https://fetchxmlbuilder.com/features/#render-flow

Get more details here

https://evolved365.com/2020/10/06/working-with-large-cds-datasets-in-power-automate-flows-and-logic-apps/

Hope it helps..

Advertisements

Posts on Azure Data Lake


How to – Access Dataverse environment storage using SAS Token

We can access our environment’s storage using a SAS token.  Download – Azure Storage Explorer – https://azure.microsoft.com/features/storage-explorer/#overview Select Connect to Azure resources option Select ADLS Gen2 container or directory for the Azure storage type. Select the Shared access signature URL for the connection To get the SAS URL – https://%5BContainerURL%5D/CDS?%5BSASToken%5D, we need ContainerURL and SASToken.…

Nishant Rana's avatar by Nishant Rana

Fixed – Azure Synapse Link for Dataverse not starting

Recently while trying to configure “Azure Synapse Link forDataverse” for one of our environments, we faced an issue, where it got stuck as shown below. Also, there were no files/folders created inside the Storage Account. https://nishantrana.me/2020/12/10/posts-on-azure-data-lake/ The accounts were properly setup – https://docs.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-synapse#prerequisites Here we had only selected contact table for sync initially, and it…

Nishant Rana's avatar by Nishant Rana

Fixed – Authorization failed. The client with object id does not have authorization to perform action ‘Microsoft.Authorization/roleAssignments/write’ over scope ‘/subcriptions’ while configuring Azure Synapse Link for Dataverse

Recently while configuring Azure Synapse Link for Dataverse, for exporting data to Azure Data Lake we got the below error – {“code”:”AuthorizationFailed”,”message”:”The client ‘abc’ with object id ‘d56d5fbb-0d46-4814-afaa-e429e5f252c8’ does not have authorization to perform action ‘Microsoft.Authorization/roleAssignments/write’ over scope ‘/subscriptions/30ed4d5c-4377-4df1-a341-8f801a7943ad/resourceGroups/RG/providers/Microsoft.Storage/storageAccounts/saazuredatalakecrm/providers/Microsoft.Authorization/roleAssignments/2eb81813-3b38-4b2e-bc14-f649263b5fcf’ or the scope is invalid. If access was recently granted, please refresh your credentials.”} As well…

Nishant Rana's avatar by Nishant Rana

How to – Export Dataverse (Dynamics 365) data to Azure SQL using Azure Data Factory pipeline template

[Visual Guide to Azure Data Factory – https://acloudguru.com/blog/engineering/a-visual-guide-to-azure-data-factory%5D Using the new Azure Data Factory pipeline template – Copy Dataverse data from Azure Data Lake to Azure SQL – we can now easily export the Dataverse data to Azure SQL Database. https://docs.microsoft.com/en-us/power-platform-release-plan/2021wave1/data-platform/export-dataverse-data-azure-sql-database Check other posts on Azure Data Factory Select Pipeline from template option inside the…

Nishant Rana's avatar by Nishant Rana

Fixed – AuthorizationFailed. The client with object id does not have authorization to perform action ‘’Microsoft.Authorization/roleAssignments/write’ over scope ‘storageaccount’ – Azure Data Lake

While configuring the Azure Synapse Link/ Export to Data Lake service, we were getting below error for one of the users. {“code”:”AuthorizationFailed”,”message”:”The client ‘nishantr@pmaurua105.onmicrosoft.com’ with object id ‘d56d5fbb-0d46-4814-afaa-e429e5f252c8’ does not have authorization to perform action ‘Microsoft.Authorization/roleAssignments/write’ over scope ‘/subscriptions/30ed4d5c-4377-4df1-a341-8f801a7943ad/resourceGroups/RG/providers/Microsoft.Storage/storageAccounts/saazuredatalakecrm/providers/Microsoft.Authorization/roleAssignments/2eb81813-3b38-4b2e-bc14-f649263b5fcf’ or the scope is invalid. If access was recently granted, please refresh your credentials.”} The current…

Nishant Rana's avatar by Nishant Rana

How to setup – Azure Synapse Link – Microsoft Dataverse

Azure Synapse Link (earlier known as Export to Data Lake Service) provides seamless integration of DataVerse with Azure Synapse Analytics, thus making it easy for users to do ad-hoc analysis using the familiar T-SQL with Synapse Studio, build Power BI Reports using Azure Synapse Analytics Connector or use Azure Spark in Azure Synapse for analytics.…

Nishant Rana's avatar by Nishant Rana

Advanced configuration settings – Azure Synapse Link / Export to Data Lake service (Dataverse/ Dynamics 365)

The Export to Data Lake service now has some Advanced configuration settings available. To learn more on Export to Data Lake service https://nishantrana.me/2020/12/10/posts-on-azure-data-lake/ The new settings allow us to configure how the DataVerse / CRM table data is written to Azure Data Lake. In-Place update or upsert (default) Append Only With the in-place update, the…

Nishant Rana's avatar by Nishant Rana

DSF Error: CRM Organization cannot be found while configuring Azure Synapse Link / Export to Data Lake service in Power Platform

Recently while trying to configure the Export to Data Lake service from the Power Apps maker portal, we got the below error. DSF Error: CRM Organization <Instance ID> cannot be found. More on configuring Export to Data Lake service – https://nishantrana.me/2020/12/10/posts-on-azure-data-lake/ The user through which we were configuring had all the appropriate rights. https://docs.microsoft.com/en-us/powerapps/maker/data-platform/export-to-data-lake#prerequisites The…

Nishant Rana's avatar by Nishant Rana

How to – Migrate Dataverse environment to a different location within the same Datacentre region – Power Platform

When we create an environment in the Power Platform admin center, we get the option of specifying the datacenter region, but not the location within it. Find the Data Center Region / Location of your Dataverse Environment- https://nishantrana.me/2021/04/27/finding-the-datacenter-region-location-of-the-microsoft-dataverse-environment/ E.g. we have specified Region as the United Arab Emirates. Now within the UAE region, we had…

Nishant Rana's avatar by Nishant Rana

Azure Data Lake Storage Component in KingswaySoft – SSIS

Download and install the SSIS Productivity Pack https://www.kingswaysoft.com/products/ssis-productivity-pack/download/ Drag the Azure Data Lake Storage Source component in the data flow Double click and click on New to specify the connection Provide the connection details and test the connection It supports both Gen 1 and Gen 2 Supports the below Authentication modes Inside the Azure Data…

Nishant Rana's avatar by Nishant Rana

Use query acceleration to retrieve data from Azure Data Lake Storage

Few key points about query acceleration – Query acceleration supports ANSI SQL like language, to retrieve only the required subset of the data from the storage account, reducing network latency and compute cost. Query acceleration requests can process only one file, thus joins and group by aggregates aren’t supported. Query acceleration supports both Data Lake…

Nishant Rana's avatar by Nishant Rana

Snapshot in Azure Data Lake (Dynamics 365 / CDS) – Azure Synapse Link

In the previous post, we saw how to export CDS data to Azure Data Lake Gen 2 https://nishantrana.me/2020/09/07/export-data-from-common-data-service-to-azure-data-lake-storage-gen2/ Here let us have a look how the sync and snapshot work. We have already done the configuration and have synced the Account and Contact entity. As the diagram depicts – there is initial sync followed by…

Nishant Rana's avatar by Nishant Rana

Error – We don’t support the option ‘HierarchicalNavigation’. Parameter name: HierarchicalNavigation when trying to load table in Power BI Desktop using Azure Data Lake Storage Gen 2 CDM Folder view (beta)

While trying to connect to a table within Azure Data Lake Storage Gen2 through CDS Folder View we got the below error Users have reported this issue with the August 2020 Update of Power BI Desktop. As suggested in the forums, downgrading to June 2020 Update fixed the issue for us. Check out Export CDS data…

Nishant Rana's avatar by Nishant Rana

How to – Use Azure Synapse Link – Export data from Common Data Service to Azure Data Lake Storage Gen2

Azure Data lake store gen 2 can be described as a large repository of data, structured or unstructured built on top of Azure Blob storage, that is secure (encryption – data at rest), manageable, scalable, cost-effective, easy to integrate with. Also check out – setup Azure Synapse with Dataverse – https://nishantrana.me/2021/06/16/how-to-setup-azure-synapse-link-microsoft-dataverse/ Export to Data Lake…

Nishant Rana's avatar by Nishant Rana

Comparing the three main ways to get Dynamics 365 data into Power BI


Ulrik (CRM Chart Guy)'s avatarcrm chart guy

There are a few different ways to get your Dynamics 365 data into Power BI. Now that the Common Data Service connector has come out of preview and is generally available, I figured it would be a good time to have a look at the different methods and how they stack up.

The different direct methods are:

  • Dynamics 365 Online connector
  • Common Data Service (CDS) Connector
  • FetchXML (via Power Query Builder for XrmToolBox)

You can also use OData directly, but since this option doesn’t have any benefits over the other three, I am going to leave it out. The Data Export Service does have some benefits, but I will have to leave that for a different blog post.

The Dynamics 365 Online connector is probably the most commonly used option as it is the result you get when searching for “Dynamics 365” in Power BI’s Get Data dialog.

D365 legacy connector

The Common…

View original post 2,494 more words

Working with large CDS datasets in Power Automate Flows and Logic Apps


Thanura Wijesiriwardena's avatarEVOLVED365

During the last 12 months I have been busy working on projects and lost my way a little bit on writing and sharing my experiences with the community. However on a positive note I learned a lot of new things during that period and now I’ve got some great content planned and waiting to be shared with you all.

Some of those learnings were around working with large CDS datasets in Microsoft Power Automate Flows and Azure Logic Apps. Here are some of those easy to learn tips, tricks, guidelines and limitations that will help you for sure.

Connecting to a Common Data Service (CDS) environment

There are two main connectors you can use. “Common Data Service” connector and “Common Data Service (current environment)” connector. Let’s look at few points to consider when selecting a connector for your workflow.

Common Data Service connectorCommon Data Service (current…

View original post 1,303 more words

User multiplexing- SSIS (KingswaySoft) + Dynamics 365 CE / CDS / Dataverse


Must read article on managing API limits

Fast data management in a limited CDS world
Thanks – Gustaf Westerlund

Creating Better PCF Component – Part 2


temmyraharjo's avatarTemmy Wahyu Raharjo

When I was writing this post, I felt the environment that I set up did not enhance my ability to write code. The reason for this is because I use jsdom, which is a mock object for the HTML component. Because it is not a real object of HTML, my focus has been changed to the test code instead of writing the implementation code. 

I ask my friend about how it is hard to implement unit testing in PCF Development (I need to install various NPM Packages to do the testing only. Most of the time because standard HTML API is not there!). Then he replied that Node.Js is not the same as Javascript in HTML. It means that PCF runs in Node.js for development, but we rely on browsers instead of Node.js functions. So my focus is set once again to make it better. We…

View original post 649 more words