Microsoft Dynamics 365 – Nishant Rana's Weblog

Easily set up Dynamics 365 CE \ CRM replication (incremental) to Azure SQL / SQL On-Premise using Skyvia’s Data Integration services

In the earlier post, we saw how we can use Skyvia’s query feature to execute SQL command against our Dynamics 365 CE

https://nishantrana.me/2019/10/25/execute-sql-select-insert-update-and-delete-command-on-dynamics-365-customer-engagement-data-using-skyvia-query/

In this post, we will cover how we can use Skyvia’s replication feature to create a copy of our cloud data i.e. Dynamics 365 Online data in a relational database like SQL Server and most interestingly keep it up to date automatically with basic configuration steps.

Creating replication of cloud data can be useful in different data integration scenarios and most importantly for reporting and data analysis need as with online we are limited with the use of Fetch XML.

The Skyvia’s replication service takes care of syncing Create, Update and Delete operation performed on Dynamics 365 Entity records with the destination SQL Database.

For this we need to Skyvia expects an object’s key as well as modifiedon, createdon fields to be included for replication.

For delete to work it expects, audit history to be enabled on that Entity.

In this post, we will take a simple example of replicating Contact entity records from Dynamics 365 Online to SQL Azure.

https://app.skyvia.com/#/packages

Create the connections

Source Dynamics CRM Connection:

Destination SQL Azure Connection:

We have a database named SampleDB already created in SQL Azure.

With connections created, now let us create a new package of type replication

For the replication, we have specified source connection as the CRM connection and destination as the SQL Azure.

We have selected the option of Incremental Updates along with Create Tables and Drop Tables option which will delete if there is an existing table of the same name as the entity and will create a new one.

In the Select Objects section, we have a specified Contact entity. We can also select multiple entities for replication.

Edit task option in the selected entity let us specify the fields and filter criteria on the source.

We also have the option of scheduling that we will look at in the end of the post.

Save the package and click on Run to see it in action.

While it is running, the Run History section shows the details

It auto refreshes with all the details.

We can see the 228 records of contact records replicated in SQL Azure.

Now let us create a new contact and run the package again.

Here we have created a contact record as shown below in CRM.

On running the package, we can see that it has added a new record in the Run History

Back in SQL Azure also our count has increased to 229.

Now let us update one of the records in the source and run the package. The run history shows one of the records being updated.

Finally, let us delete a couple of records (9) from the source and run the package. (We need to make sure we have enabled Audit on Contact entity for delete to work)

Back in SQL Azure, we can see total record count decreased

Here we were running the package manually, however, in real-world scenarios we would be scheduling it.

For this, we can edit our package and navigate to the Schedule section and click on Set Schedule.

For e.g. we have scheduled here it to run every hour starting from 6 p.m. till 10 a.m.

On save, we can check the run history

Thus, we saw how easy it is to replicate the data from Dynamics 365 CE to our SQL Database using Skyvia’s Integration service.

To achieve the same using Scribe Online Replication Service

https://nishantrana.me/2019/01/09/using-scribe-online-replication-services-rs-to-replicate-dynamics-ce-data/

and KingswaySoft

http://www.kingswaysoft.com/blog/2017/04/04/How-to-Retrieve-Incremental-Changes-from-CRM-in-your-ETL-Process

Hope it helps..

Perform actions on the host model-driven form from embedded canvas app using ModelDrivenFormIntegration control in Dynamics 365

Let us continue our previous example of embedding the canvas app within the model-driven app and extend it to perform actions on the host form using ModelDrivenFormIntegration control.

This ModelDrivenFormIntegration control acts as a bridge and brings the contextual data from the host model-driven form to the embedded canvas app.

Here we will look at some of the pre-defined actions that can be performed using the ModelDrivenFormIntegration Control.

For this update our existing Canvas App and add the below buttons to the Form control in the canvas app

Save Form

For the Save Form button, first, create the flow (Power Automate) which will update the record.

https://nishantrana.me/2019/11/04/step-by-step-calling-flow-from-powerapps-dynamics-365-ce-crm/

Below we are calling the Flow from Save Form button and passing GUID of the record along with values of name, city and phone field.

Here we have used ModelDrivenFormIntegration’s Item property to retrieve the id of the account record in our formula that is being used to call the Flow (Power Automate)

Result –

updating the address 1 city field by appending text “Redmond” in the embedded canvas app.

save

Refresh Form

The true and false parameter value of the RefershForm method specifies whether to show prompt or not before saving any unsaved data on the host form.

Result –

refreshing the form, shows the address 1 city field of the form in the right section with the appended text “Redmond” as saved earlier.

refresh

Navigate to Main Form

NavigateToMainForm method’s parameters are entity name, form name, record id.

Result –

Here we are opening the same record.

navigatemain

Navigate to View

NavigateToView method’s parameters are entity name and view name.

Result –

Opens the view specified in the parameter

NavigateToView

Open Quick Create Form

NavigateToView method’s parameter is entity name.

Result –

Opens the quick create form of the entity specified in the parameter.

Get all the details here

Hope it helps..

KingswaySoft SSIS Integration Toolkit for Microsoft Dynamics 365 vs using Data Import Wizard

I was recently asked that what are the benefits of using KingswaySoft tools for Data Migration need vs our out of the box Data Import wizard.

We are talking about 2 different solutions altogether so the direct comparison might not be appropriate. However, highlighting a few of the capabilities and limitations of each of them.

As we all know, the simplest and straight forward way of importing data into Dynamics 365 is by using the Data Import Wizard.

A Dynamics CRM user can be trained to use this feature.

The feature is easy to use, intuitive and does the job well when it comes to quickly import a small set of data.

Also, we can see the status of the import job, did it succeed or how many records failed, etc.

However, there are certain limitations

We cannot term it as a data integration solution as we are not scheduling it.

It is not real-time as the asynchronous job running in the background takes care of it.

We can only perform an update using the file exported one time only. Every time we are performing an update, we need to export the records again. We will get an error if the records have been modified after the file was exported.

The status of the records cannot be changed.

We cannot have our custom logic or transformation is applied.

The maximum file size for .zip file is 32 MB.

For other file formats like .csv,.xml or text files the limit is 8 MB.

Microsoft recommends limiting import to 20K rows or fewer. For a large volume of data Microsoft recommends programmatic way as most efficient.

Here another option for data import that we can use is the Configuration Migration Tool

https://docs.microsoft.com/en-us/power-platform/admin/manage-configuration-data

However, we normally use third party tool like the one developed by KingswaySoft for most of our complex data migration requirements, instead of writing one such tool ourselves because it would need to develop from scratch with performance, logging, ease of configuration etc. in mind, which could be time-consuming.

Below we can find all the details about the SSIS Integration toolkit developed by KingswaySoft

https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365

With powerful components like CRM Connection Manager, CRM Source and Destination Component and CRM OptionSet mapping component we can develop codeless data integration solutions (packages).

With SSIS coming into the picture we are not limited to XML, Excel, CSV or text files as our source. Recently in one of our implementations, we had complex XML as our source files, for which we use the XML Source Component of KingswaySoft

https://www.nigelfrank.com/blog/integrate-dynamics-365-for-customer-engagement-with-xml/

The components are intuitive, flexible, feature-rich and very easy to use and configure. We can easily map our source data with Dynamics 365 entities.

We can have full logging (failure and success) and retry mechanism designed in the SSIS Package easily.

With KingswaySoft CRM components we can achieve high performance. The batch size and the total number of threads can be configured.

KingswaySoft offers the free developer license, under this license the software can be used within Visual Studio or Business Intelligence Development Studio. The limitation is that it cannot be run on a scheduled basis or from the command line outside the development environment, for that one needs a license.

https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365/purchase

The tool is constantly evolving along with new updates and changes in the Dynamics 365 CE /CDS.

https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365/change-log

The solution built using KingswaySoft can be deployed to Azure-SSIS Integration runtime.

The KingswaySoft’ s support team is very knowledgeable and helpful and we can expect a quick resolution of any of our queries. I remember our team facing difficulty while configuring one of their components and the support team was able to guide us on how to configure it correctly.

However, there are few things to consider while opting for KingswaySoft

Need for SQL Server Environment (for scheduling the packages)
Resource(s) skilled in SSIS.

The other tools worth considering when it comes to data integration and migration need with Dynamics 365 are

Scribe

https://help.scribesoft.com/scribe/en/sol/conn/crm.htm

Cozy Roc

https://www.cozyroc.com/ssis/dynamics-crm

Below are couple of blog posts I wrote while working with the KingswaySoft’s components and Scribe Online

https://nishantrana.me/tag/kingswaysoft/

https://nishantrana.me/category/scribe-online/

Hope it helps..

Embedding a canvas app on a model-driven app in Dynamics 365

Let us start with some of the key points about the embedded canvas app

The embedded canvas app is now generally available. During preview.
The embedded canvas app supported sub grid control in preview, which is now no longer available.
The canvas app custom control is only supported for the Unified Interface Web client type. It doesn’t support Phone and Tablet client types.
We can only have 1 canvas app on the form. We can have multiple embedded canvas app but only 1 of them could be enabled.
It is recommended to have an embedded canvas app on a required field else it will not refresh on any change in the data on the host model-driven form.
The embedded app can only be created from the host model-driven form.
The existing canvas app cannot be used as an embedded canvas app.

To create the canvas app custom control, select the controls tab in the Field Properties dialog, and click on Add Control.

Select Canvas app in App Control dialog.

The Canvas app will have Web client selected as default and Phone and Tablet option will be greyed out.
Entity Name shows the name of the field that is associated with it.
App ID will be populated when the app is created.

Clicking on Customize opens the PowerApps Studio.

We can see the canvas app form control added to the canvas app.

We can see the fields for city, name, and phone already added to it.

ModelDrivenFormIntegration is a special control that is responsible for bringing the contextual data from the form to the embedded canvas app.

Using Edit Fields option, we can add, remove, edit, reorder the fields of the data source.

Just for the simplicity let us keep the app as it is.

Click on File à Save à The cloud (Save to PowerApps)

On a successful save, we can see the App ID generated.

Uncheck Display label on the form, save and publish the customization.

Changing the value in the Phone field in the form and saving the form changes the value inside the canvas app form control.

To Edit the canvas form control, we need to follow the same steps, open the field properties, controls tab and click on Customize

After making the changes inside PowerApps studio, save and publish the changes.

The other Dynamics CRM User say poweruser2 will get the below error message when they open the same form if the app is not shared with them.

The app can be shared with specific users, security groups or everyone within the organization. The users should have appropriate security roles to access the entities and also appropriate permissions on the cloud services that are being used as part of the canvas app.

There are no privileges specific to the canvas app within Dynamics 365 CE

To share the app, Navigate to File and select Share option

Here we searched for the user, made him Co-owner and assigned security roles which will give him access to Accounts entity.

We can see the embedded canvas app rendered properly in this case for the poweruser2

In this post, we covered some of the key points about the embedded canvas app along with steps to create, update and share it (although we just used the OOB generated app here), I would recommend to check out the videos by Shane Young to learn about PowerApps and different scenarios that we could implement.

Hope it helps..

Step by Step – Calling Flow from PowerApps (Dynamics 365 CE/CRM)

Let us take a simple example of creating lead to understand how we can call Flow from PowerApps.

TextInput for entering the Last Name, First Name and Email ID.
Button to call Flow.
The label named GUID which will display the GUID of the created lead record.

Now select the button, go to Action menu and click on Flows to create a new flow.

Click on create a new flow.

Select Instant – from blank option

Select PowerApps as the trigger.

Add a new Step – Create a new Record (Dynamics 365)

To create a new record step, specify the Dynamics 365 Organization Name, select Leads as the entity name.

For the Last Name field, click Ask in PowerApps that will generate the parameter to which we will pass the value from our PowerApps.

The generated parameter.

Repeat the above step for the First Name and Email field.

*for Topic field we have passed a hardcoded value.

Save the create step and add a new step “Respond to a PowerApp or flow”

Here we have selected output of type Text

We have added the output parameter as varLeadId and selected leadid as the unique identifier.

Save the Flow and navigate back to PowerApp and select the flow created for our button.

After adding the Flow, the next step is to update the formula to send the field’s value to the flow and also to receive the output from it.

Below will be our formula to pass the text to the flow.

But as we are expecting the GUID in return, we need to update it.

We are using the Set function to set the variable varResponseFromFlow with varleadid output.

Also, we have updated the label field for GUID with the variable

Now let us save the changes and run the PowerApp.

On clicking the button, we can see the GUID updated on the label on the form.

Back in our Dynamics 365 CE, we can see the lead created.

Thus we saw how easy it is to write a PowerApp which interacts with Dynamic 365 CE using Flow. We took a very basic example to understand how things glue, but we can clearly see the potential here to implement complex real-world scenarios with ease.

Hope it helps..

Execute SQL Select, Insert, Update and Delete command on Dynamics 365 Customer Engagement Data using Skyvia Query

Recently came across the wonderful product called Skyvia Query

It is one of the most advanced Visual Query Builders out there, that let us write complex SQL queries with simple drag and drip and configuration. The most interesting part is that it works with both Cloud as well as On-Premise databases.

To try it out we can create a free trial account.

https://app.skyvia.com/register

In this post, we will see how we can quickly use it to connect to our Dynamics 365 CE Database and perform select, insert, update and delete queries.

Navigate to connections and create a new Dynamics CRM Connection

Provide the connection details and test and save the connection.

Navigate to the Query section and select New Builder option to use SQL Builder tool to create the query

Specify the connection and search for the Entity (table)

Here we have searched for and selected the Opportunity Entity. Now simply drag and drop the fields to the designer surface.

Results Fields:

Filters:

Sort Fields:

Click on Execute (F9) to get the result.

Let us drag and drop opportunity rating field in the result fields section and Execute it

The results can be exported to PDF, CSV and can also be used for creating a chart that can be exported as an image.

PDF:

Chart:

Going back to our designer screen, now let us work with the actual SQL query that was being generated by the query builder for us.

For this, click on the SQL tab.

Below is the generated query.

We can work with the query in the SQL query window itself and can also drag and drop fields from the Table on the left.

For e.g. we have dragged Records count on the SQL query window, which generated the count query for us.

Now let us see the Insert query in action

We can see our contact record created

Now let us perform Update.

Here it asks for the confirmation before performing the update. Selecting Apply All performs the update.

Now let us look at the DELETE query in action.

Similar to Update it asks for the confirmation before deleting the records.

Thus Skyvia Query tool makes it so easy and intuitive to work with Dynamics 365 CE data, as we are using SQL statement instead of Fetch XML.

I would highly recommend giving this product a try.

https://skyvia.com/pricing/

Hope it helps..

Using PCF Custom Control Builder (XrmToolBox Plugin) to update existing control in Dynamics 365 / PowerApps

A couple of months back I had written my first PCF Control to display GUID of the record, it was more of a hello world kind of thing.

https://pcf.gallery/guid-control/

Well, the code was using Xrm.Page client API which as we know is deprecated.

So, it was time to update it to use context instead.

Single Instance / Organization Considerations in Dynamics 365 Customer Engagement / Dynamics CRM

Let us start from the scenario that the enterprise has decided to use Microsoft Dynamics CRM /365 CE as a preferred customer relationship solution for their organization.

For a large enterprise and/or having multiple different departments, the first question that comes up is, do they need to have a single instance or organization or multiple instance or organization. Well, that decision is not that easy but one thing is for sure the decision should be made as early as possible in the project.

Well you could refer to these posts that could help us to make the decision

https://medium.com/capgemini-dynamics-365-team/does-enterprise-scale-dynamics-require-multiple-instances-f1931a443930

https://nishantrana.me/2019/07/23/single-organization-vs-multiple-organization-single-instance-vs-multiple-instance-key-points-to-consider-in-dynamics-365-ce-on-premise/

Now suppose we have made the decision to go with the single instance \ organization, so what are some of the key points we need to consider.

There could be multiple ways of implementing and defining the segregation layer as well as what needs to be shared from data and customization perspective among the different business units, the most important being the security model and the business unit structure being one of the key components of it.

Check out the below Webinar –

http://www.xrmcoaches.com/2017/07/free-webinar-friday-creating-your-dynamics-365-security-model/

also listen to the below podcast to get some guidance around the setting up the security model

http://crmmvppodcast.com/episode-19-setting-up-your-security-architecture-like-a-crm-mvp

In one of our projects, the way we segregated different departments, after defining the appropriate business unit structure, was by having the Business Unit lookup (custom field) in all the common entities that were being shared by the departments and populating that field accordingly, every time record is created either through User Interface or through code and to refer the same field in the customizations (plugin, workflow etc.) as and when applicable.

In the other project, we were referring to the Business Unit lookup field in the System User / Team Entity.

Well there could be multiple ways of doing the same with its own pros and cons.

But what if you have already implemented for one of the units and didn’t consider that the same instance would need to be extended to accommodate the different business unit(s).

Well, then it is time to modify a couple of things to make the Business Unit structure scalable (if it is not).

Let us go through the each of the main components one by one and how they can be segregated among different business units.

Wrong root business unit name?

Set Parent Business Unit lookup field as optional, open the Business Unit record and rename it.

Need to rename the organization?

For Online à

Navigate to Admin Portal, select the instance and click on edit and update the organization name and if required the URL as well as shown below.

For On-Premise à

Follow the below steps to update the Display Name

https://blogs.msdn.microsoft.com/darrenliu/2013/09/21/change-crm-organization-display-name/

Updating Organization Database Name

https://blogs.technet.microsoft.com/crmbipl/2008/01/14/how-to-change-crm-4-0-organizations-database-name/

Forms

Create separate new Main forms for different business units.

Enable appropriate security roles for the Form.

Views

We can make use of the 3^rd Party Solution à
http://www.xrmforyou.com/role-based-views.html
Name the view appropriately to reflect which Business Unit it belongs to.
Train the users to use the appropriate View and Pin It.

Fields

Reuse the fields that are common instead of ending up creating a large number of fields.

Field security profiles can also be used if required when certain entity’s records are shared across the business unit and we want to hide the sensitive information.

Business Rules

Business rule can be configured to run on the scope of Form instead of the Entity.

Dashboards

For Dashboards, we can enable the security roles.

Workflows

Use scope fields to make sure the workflows fire appropriately.
Or put the first step to check for which business unit it is firing (depends on the segregation logic implemented)

Plugins

Have a field in the records to identify to which Business Unit it belongs to and check for this field as the first thing in the plugin or check for the Business Unit of the user under whose context the plugin is getting fired or use the secure \ unsecure configuration etc. it purely depends upon the way the segregation logic has been implemented.

SLAs

Use SLA lookup field to apply the SLA dynamically through plugin or workflow.

https://nishantrana.me/2019/07/24/apply-sla-choosing-the-sla-in-the-sla-lookup-field-for-the-sla-enabled-entities-in-dynamics-365-ce/

Business Process Flow

If there are multiple business process flow, make sure appropriate one is applied

Apply business process flow while creating an entity record.

https://www.powerobjects.com/blog/2019/04/24/business-process-flows-switch-bpf-dynamics-365-javascript/

There would be multiple other key points to be considered apart from the one listed above specific to each implementation; the above points are some of the things that can be taken as guidelines.

Lastly we must consider creating different apps for different department \ teams\ users and add required solution components to it.

Design custom business apps by using the app designer

Please feel free to add comments and share your thoughts.

Hope it helps..

Business Unit Hierarchy – My First XrmToolBox Plugin

Check out – XrmToolBox Plugin to generate Business Unit Hierarchy by Phani

Phani Rajasekhar

I am very happy to announce that my first plugin “Business Unit Hierarchy” is listed in XrmToolBox now.

Thanks to Nishant Rana for giving me an idea to develop this.

This plugin is simple but very useful. Most of the times we navigate to so many windows to understand the BU hierarchy in our organization (I faced this situation many times) but with this plugin it is very easy to get the complete overview of BU Hierarchy. I plotted the BU Hierarchy in tree view presentation which is easy to understand for any user.

Here are list of steps to use this plugin