Synchronize (Bi-Directional) Dynamics 365 CRM Cloud data with SQL Azure using Skyvia’s Synchronization Package

Continuing our previous post where we used the replication package

https://nishantrana.me/2019/11/13/easily-set-up-dynamics-365-ce-crm-replication-incremental-to-azure-sql-sql-on-premise-using-skyvias-data-integration-services/

here we will see how we can use the synchronization service for bi-directional sync.

Below of the prerequisites of synchronization package to perform bi-directional sync –

  • We need to use the created on and modified on fields of the Entity
  • The synchronized tables to have an auto-generated primary key.

So here we need to alter the SQL Azure’s Contact table that was auto-created earlier by the Replication package to have the auto-generated primary key.

Connect to the SQL Azure database using Skyvia’s Query service and perform the alter operation

Or use the Query Editor (preview) of Azure

With necessary changes made, let us create a new Synchronization package

Specify the connection and click on Add Task to create the synchronization task.

We have specified contact entity from Dynamics CRM Source and contact table in SQL Azure which was earlier created for us by the replication package.

It automatically maps all the columns here based on the column name.

We can also manually specify mapping here for both Source To Target and Target to Source fields to be used for sync.

Click on Finish once done with the mapping.

Similarly to replication packages, the synchronization packages can be scheduled.

Click on Save to create the Task.

Click on Run

The run history shows the 220 records synced from Dynamics CRM to SQL Azure.

We can see the 220 records created in our SQL Azure DB

Now let us update a record in SQL Azure and run the package again.

We can see the record updated in Dynamics CRM

Now let us delete a few records from Azure SQL and run the package

We can see the records delete from Dynamics CRM

Similarly, let us delete a couple of records from Dynamics CRM and run the package

We can see the 8 records deleted in the destination SQL Azure

Thus, we saw how easy it is to do bi-directional sync using Skyvia’s Synchronization package.

Hope it helps..

Single Instance / Organization Considerations in Dynamics 365 Customer Engagement / Dynamics CRM

Let us start from the scenario that the enterprise has decided to use Microsoft Dynamics CRM /365 CE as a preferred customer relationship solution for their organization.

For a large enterprise and/or having multiple different departments, the first question that comes up is, do they need to have a single instance or organization or multiple instance or organization. Well, that decision is not that easy but one thing is for sure the decision should be made as early as possible in the project.

Well you could refer to these posts that could help us to make the decision

https://medium.com/capgemini-dynamics-365-team/does-enterprise-scale-dynamics-require-multiple-instances-f1931a443930

https://nishantrana.me/2019/07/23/single-organization-vs-multiple-organization-single-instance-vs-multiple-instance-key-points-to-consider-in-dynamics-365-ce-on-premise/

Now suppose we have made the decision to go with the single instance \ organization, so what are some of the key points we need to consider.

There could be multiple ways of implementing and defining the segregation layer as well as what needs to be shared from data and customization perspective among the different business units, the most important being the security model and the business unit structure being one of the key components of it.

Check out the below Webinar –

http://www.xrmcoaches.com/2017/07/free-webinar-friday-creating-your-dynamics-365-security-model/

also listen to the below podcast to get some guidance around the setting up the security model

http://crmmvppodcast.com/episode-19-setting-up-your-security-architecture-like-a-crm-mvp

In one of our projects, the way we segregated different departments, after defining the appropriate business unit structure, was by having the Business Unit lookup (custom field) in all the common entities that were being shared by the departments and populating that field accordingly, every time record is created either through User Interface or through code and to refer the same field in the customizations (plugin, workflow etc.) as and when applicable.

In the other project, we were referring to the Business Unit lookup field in the System User / Team Entity.

Well there could be multiple ways of doing the same with its own pros and cons.

But what if you have already implemented for one of the units and didn’t consider that the same instance would need to be extended to accommodate the different business unit(s).

Well, then it is time to modify a couple of things to make the Business Unit structure scalable (if it is not).

Let us go through the each of the main components one by one and how they can be segregated among different business units.

Wrong root business unit name?

Set Parent Business Unit lookup field as optional, open the Business Unit record and rename it.

Need to rename the organization?

For Online à

Navigate to Admin Portal, select the instance and click on edit and update the organization name and if required the URL as well as shown below.


For On-Premise à

  • Follow the below steps to update the Display Name

https://blogs.msdn.microsoft.com/darrenliu/2013/09/21/change-crm-organization-display-name/

  • Updating Organization Database Name

https://blogs.technet.microsoft.com/crmbipl/2008/01/14/how-to-change-crm-4-0-organizations-database-name/

Forms

Create separate new Main forms for different business units.

Enable appropriate security roles for the Form.

Views

Fields

Reuse the fields that are common instead of ending up creating a large number of fields.

Field security profiles can also be used if required when certain entity’s records are shared across the business unit and we want to hide the sensitive information.

Business Rules

Business rule can be configured to run on the scope of Form instead of the Entity.

Dashboards

For Dashboards, we can enable the security roles.

Workflows

  • Use scope fields to make sure the workflows fire appropriately.
  • Or put the first step to check for which business unit it is firing (depends on the segregation logic implemented)

Plugins

Have a field in the records to identify to which Business Unit it belongs to and check for this field as the first thing in the plugin or check for the Business Unit of the user under whose context the plugin is getting fired or use the secure \ unsecure configuration etc. it purely depends upon the way the segregation logic has been implemented.

SLAs

Use SLA lookup field to apply the SLA dynamically through plugin or workflow.

https://nishantrana.me/2019/07/24/apply-sla-choosing-the-sla-in-the-sla-lookup-field-for-the-sla-enabled-entities-in-dynamics-365-ce/

Business Process Flow

If there are multiple business process flow, make sure appropriate one is applied

Apply business process flow while creating an entity record.

https://www.powerobjects.com/blog/2019/04/24/business-process-flows-switch-bpf-dynamics-365-javascript/

There would be multiple other key points to be considered apart from the one listed above specific to each implementation; the above points are some of the things that can be taken as guidelines.

Lastly we must consider  creating different apps for different department \ teams\ users and add required solution components to it.

Design custom business apps by using the app designer

Please feel free to add comments and share your thoughts.

Hope it helps..

Generate Dynamics 365 CE (CRM) Business Unit Org Chart through SmartDraw

Recently we were looking at the options of generating a Business Unit Org Chart for quick review.

Using SmartDraw we were able to do achieve it very quickly.

Steps

  • Keep only the Name and Parent Business column and delete all the other columns including the hidden ones.

  • Excel with required columns.

  • Create the trial account of SmartDraw cloud

  • Click on Org Charts from the New Document menu.

  • Select Build Org Chart from Data option

  • Browse and select the xls file.

  • Click on Process.

  • Select Show in Shape for the columns and click on the Build diagram.

  • The result

  • The other option is to select any of the Organizational Charts

  • Select Import from file

  • Select the excel file

  • The result

  • The chart can be exported, shared and print.

Thus SmartDraw makes it very easy to generate the Org Chart for the Business Unit.

To achieve the same through Visio

http://www.planetxrm.com/12-steps-to-creating-crm-business-unit-org-charts/

Hope it helps..

You do not have the necessary privileges to import security roles. Microsoft.Crm.CrmSecurityException: SecLib::CheckPrivilege failed while importing solution in Dynamics 365 CE

Recently after upgrade of our Dynamics 365 CE from 8.2 to 9.1, when the System Administrator tried importing the solution (managed), he got the below error

You do not have the necessary privileges to import security roles. : Microsoft.Crm.CrmSecurityException: SecLib::CheckPrivilege failed. User: e7282250-7d82-e711-80df-1458d0431690, PrivilegeName: prvWriteRole, PrivilegeId: bd123e14-17ba-40f6-8d8b-18f4bffa7e50, Required

The error message is quite weird, first as we have never got this error earlier and here it was System Administrator user who was importing.

We tried importing the solution with the using another System Adminstrator account and it worked for that other user.

Hope it helps..

Requesting the backup of Microsoft Dynamics 365 Online Database

In theory, requesting the backup of the online database seems very easy and straightforward. We basically need to raise a support ticket with Microsoft and provide the following details.

  • SQL Version – Standard or Enterprise
  • SQL Year of the version. In our case it was (Microsoft SQL Server 2016 (SP1) (KB3182545) – 13.0.4001.0 (X64) Oct 28 2016 18:17:30 Copyright (c) Microsoft Corporation Standard Edition (64-bit) on Windows Server 2016 Datacenter 6.3 <X64> (Build 14393)
  • Global Admin E-mail Address – to who the backup will be shared.

We had requested the same, some 3 weeks back. In our case, the Database size of our production was around 600 GB out of which 200 GB were occupied by Audit Data. For some business-specific reason, we wanted to have the backup of Audit Data with us before we delete it. Now, this Audit has a separate story with it, when we tried deleting it from UI we got the SQL Time Out issue and on raising the support ticket when Support Team tried the same through back end it, resulted in our Production Server being down briefly for few minutes. We also tried to take the back up through 3rd party tools using SSIS, however again we ran into some issue where the tool was not able to pull the Audit Data for few of the entities.

Well to cut the story short, it is around 21 days after we raised the request, and we are yet to receive the backup of our production database, we have been told that support team is actively working on it and it is because of the large size of the organization.

Will keep posting the update on the backup and the Audit deletion.

The purpose of this blog post was to share that certain things might seem straightforward and would have taken maximum few hours, could still run into issues and challenges causing delays. So, when we plan for it we need to aware of these factors as well.

Hope it helps..

Sample Code to retrieve instances using Online Management API in Dynamics 365 Customer Engagement

With version 9.0 of Dynamics 365 CE we now have Online Admin API that supports the following operations like Create, Retrieve, Delete, Backup and Restore instances. The user needs to have the Global Administrator or Service Administrator role in the Office 365 tenant to perform these operations.

As a first step we need to register the application with Azure Active Directory.

https://nishantrana.me/2016/11/13/register-a-dynamics-365-app-with-azure-active-directory/

and get the values for

Client Id and Authority.

Get the Service URL here

https://docs.microsoft.com/en-us/dynamics365/customer-engagement/developer/online-management-api/get-started-online-management-api#service-url

Sharing a sample code which can be used to retrieve all the instances in a specific tenant.

The sample code: –


public class MySampleApp
{

private const string Resource = "https://admin.services.crm.dynamics.com/";

// username and password
private const string UserName = "username.onmicrosoft.com";
private const string Password = "password";

// client id and authority of the application registered
private const string ClientId = "b2c5028d-57e6-4df7-9940-3243214948b";
private const string Authority = "https://login.microsoftonline.com/3432432e2-5fbe-4b78-a55a-bcb342d4f859/";

private static AuthenticationResult _authResult;

private static void Main(string[] args)
{
var authContext = new AuthenticationContext(Authority, false);
var credentials = new UserCredential(UserName, Password);

// Get the token
_authResult = authContext.AcquireToken(Resource, ClientId, credentials);

Task.WaitAll(Task.Run(async () => await GetInstances()));
}

private static async Task GetInstances()
{
var httpClient = new HttpClient
{
BaseAddress = new Uri(Resource),
Timeout = new TimeSpan(0, 2, 0)
};

httpClient.DefaultRequestHeaders.Add("OData-MaxVersion", "4.0");
httpClient.DefaultRequestHeaders.Add("OData-Version", "4.0");
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer",
_authResult.AccessToken);

var retrieveResponse =
await httpClient.GetAsync("/api/v1/instances");

if (retrieveResponse.IsSuccessStatusCode)
{
var jRetrieveResponse =
retrieveResponse.Content.ReadAsStringAsync().Result;

var result = JArray.Parse(jRetrieveResponse);
foreach (var data in result)
Console.Write("Id " + data["Id"] + "\nUnique Name " + data["UniqueName"]);
}
}
}

Result: –

The other properties: –

SiteMapName in the AppModuleSiteMap is null or empty error while importing V9 Solution in Dynamics 365 Customer Engagement

We recently upgraded our Dev and Test environment to V 9.0.

Dev

Test

While importing a solution from Dev to Test which had SiteMap we got the below issue.

“The SiteMapName in the AppModuleSiteMap is null or empty”

It turns out that the following tags were required and were missing in the Customization.xml

We added that tag and were able to import the solution successfully.

Interestingly if we export the same solution from Test and search for this tag, it is again missing, however, we can see the following XML section added there

This section was not there in dev’s sitemap.

Hope it helps..

Plugin on Pre-Validation Stage in Dynamics 365 CE

Recently we had a requirement to delete the Account record without deleting the associated Contact records.

If we try deleting the account record we’d get the following message box

The relationship definition can’t be updated as well to achieve this

So, we wrote a plugin on the pre-validation stage of pre-delete event of Account, which will retrieve and loop through all the child contact records and set its parent customer field as null.

In case of pre-operation the child records were not available.


public void Execute(IServiceProvider serviceProvider)
{
ITracingService tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
IPluginExecutionContext pluginContext = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
IOrganizationService organizationService = serviceFactory.CreateOrganizationService(pluginContext.UserId);
EntityReference targetEntity = (EntityReference)pluginContext.InputParameters["Target"];
QueryExpression getContacts = new QueryExpression("contact");
getContacts.Criteria.AddCondition(new ConditionExpression("parentcustomerid", ConditionOperator.Equal, targetEntity.Id));EntityCollection allContacts = organizationService.RetrieveMultiple(getContacts);foreach (Entity contact in allContacts.Entities)
{
Entity contactToBeUpdated = new Entity("contact");
contactToBeUpdated.Id = contact.Id;
contactToBeUpdated.Attributes["parentcustomerid"] = null;
organizationService.Update(contactToBeUpdated);
}}

 

Another practical scenario

https://www.inogic.com/blog/2017/03/plugin-pre-validation-operation-to-show-an-error-message-as-well-as-log-the-error/

Hope it helps..

Upgrade to new Xrm client API object model (v9) smoothly using XrmToolBox plugins

We are currently in process of upgrading Dynamics from version 8.2 to 9.0. One of the major change is updating our current JavaScript to the new Xrm Client API Object Model.

https://docs.microsoft.com/en-us/dynamics365/get-started/whats-new/customer-engagement/important-changes-coming#some-client-apis-are-deprecated

https://docs.microsoft.com/en-us/dynamics365/customer-engagement/developer/clientapi/understand-clientapi-object-model

Below are the most useful tools à

Install the plugin, connect to your organization and click on Retrieve Jscript Webresources, it will list all the JScript Web resources found.

Click on Scan

It will list down all the issues found.

One common change that we need to do across all our JScript is to pass the ExecutionContext from Form as well as Fields (and Ribbon) events. Script Finder makes it extremely easy to find them out.

Click on Find Scripts usage and it will generate a report with all the details as shown below

Now we can follow the generated report and directly make the change to the form, field, and ribbon instead of searching for it manually.

Happy Upgrading !!

Upgrading from Dynamics CRM 2016 On-Premise to Dynamics 365

Make sure the Solution is compatible to Dynamics 365 (8.2), for this check the Version Compatibility

https://docs.microsoft.com/en-us/previous-versions/dynamicscrm-2016/developers-guide/gg328109%28v%3dcrm.8%29#version-compatibility

Upgrade the existing CRM 2016 On-Premise Server to CRM 2016 (8.1) for its solution to be compatible with Dynamics 365 Online using cumulative updates.

https://support.microsoft.com/en-gb/help/3142345/microsoft-dynamics-365-onpremise-cumulative-updates

If upgrading from previous version i.e. prior to CRM 2016, use “Migrate by using a new instance of Microsoft SQL Server” approach

https://technet.microsoft.com/en-us/library/hh699669.aspx

https://technet.microsoft.com/en-us/library/hh699747.aspx

Once updated to compatible CRM 2016 On Premise (8.1), go to settings à customizations and export the existing solutions or create a new solution, add all the required solution components to it as unmanaged which will be imported to the new Dynamics 365 online environment.

Create a new Dynamics 365 Online Environment and synchronize it with your existing Active Directory

https://blogs.msdn.microsoft.com/crm/2013/07/18/how-to-synchronize-crm-online-with-your-active-directory/

With solution ready and imported to new Dynamics 365 Online, next step is migration of data.

Consider using a data migration tool like KingswaySoft.

Check out their Migration Starter Pack.

https://www.kingswaysoft.com/blog/2016/09/16/Announcing-Migration-Starter-Pack-for-CRM-Online-and-CRM-On-Premise

or Scribe

https://www.scribesoft.com/solutions/dynamics-365/

Use following tools to update the JavaScript to Version 9.0 of XrmToolBox.

https://www.xrmtoolbox.com/plugins/XrmToolBox.Dynamics365V9JavascriptValidator/

  • Script Finder (to update the form and fields for passing the executionContext)

https://www.xrmtoolbox.com/plugins/MsCrmTools.ScriptsFinder/

https://community.dynamics.com/crm/b/develop1/archive/2017/11/11/executioncontext-hits-the-big-time

Update JavaScript to use Web API.

https://github.com/jlattimer/CRMRESTBuilder

SQL Based report, if possible, needs to be converted to use Fetch XML. Use the following tool to speed up the process.

http://www.sql2fetchxml.com/

If the reports are too complex, plan to use Power BI reports.

https://technet.microsoft.com/en-us/library/dn708055.aspx

Plugin and Custom workflow needs to be updated to run in an isolated environment i.e. sandbox along with references to the latest SDK assemblies.

https://docs.microsoft.com/en-us/dynamics365/customer-engagement/developer/plugin-isolation-trusts-statistics#web-access

https://www.nuget.org/packages/Microsoft.CrmSdk.CoreAssemblies/

If the existing On-Premise is integrated to other applications, make sure they can still talk with new Dynamics 365 Online.

Recurring jobs if needed, can be deployed to Azure as Azure WebJobs.

https://docs.microsoft.com/en-us/azure/app-service/websites-dotnet-deploy-webjobs

Use Hybrid Connection to talk with On-Prem resources.

https://docs.microsoft.com/en-us/azure/app-service/app-service-hybrid-connections

These were some of the points I could quickly think of. Please share your thoughts and experiences in comments. I’d update this list.

Do check out :

Microsoft Dynamics CRM (on-premises) to Microsoft Dynamics
Online Migration Guide

https://download.microsoft.com/download/6/D/6/6D67BDEA-1D67-42B4-A52A-CF13CD547CB5/OPtoCRMOnlineMigration.pdf

Hope it helps..