Process Optimization – Dynamics 365 CE / CRM / CDS SSIS Integration toolkit – KingswaySoft


Process Optimizations is a new feature added in the CDS / CRM Destination Component of KingswaySoft’s SSIS Integration Toolkit as part of November 2020 Release.

Check other posts on SSIS and Dynamics 365

https://nishantrana.me/2020/10/16/ssis-kingswaysoft-and-dynamics-365/

These options if enabled, will turn off or disable plugin, workflow, entity auditing during pre-execution temporarily, during the writing process, and will revert the changes to the original state after successful execution.

  • Tentatively Disable Relevant Plugins (if any)
  • Tentatively Disable Relevant Workflows (if any)
  • Tentatively Disable Relevant Auditing (if any)

Let us take a simple example to see it in action.

Here we have a simple data flow that creates contact records inside CDS using Data Spawner.

https://nishantrana.me/2020/05/26/using-data-spawner-component-ssis-to-generate-sample-data-in-dynamics-365/

Let us run the package and see the status of the plugin, workflow, and audit with regards to the contact entity.

As expected, we do not see any Audit records created for contact records creation.

We see an event where Audit is disabled and then enabled later for the contact entity, during the duration of package running and after its successful completion.

We can also see our workflow in status – Draft

In the case of Plugin, we didn’t see the step being disabled and it was still being triggered

As per the documentation –

Get the toolkit here

https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365

Hope it helps..

Changes in CDS / CRM Destination Component – SSIS Integration Toolkit for Dynamics 365


With version 20.2 November 2020 release, there have few updates added to the KingwaySoft’s CDS/CRM Destination Component, few are changes in the label and others are metadata changes, that one should be aware of before updating.

Check Post on SSIS and Dynamics 365

https://nishantrana.me/2020/10/16/ssis-kingswaysoft-and-dynamics-365/

  • Label changes.

Version 20.1 –

Version 20.2 –

  • CrmRecordId renamed to SavedRecordId

Version 20.1 –

Version 20.2 –

  • CrmErrorMessage renamed to ErrorMessage

Version 20.1 –

Version 20.2 –

Thus, any package saved with the new version will not work with the older version, and it is recommended to take full back up of the package before updating to the new version.

Hope it helps..

Extract attachment from Notes in Dynamics 365 / CE / CDS – SSIS (KingswaySoft)


Recently we were exploring ways of extracting attachment from notes to a local drive for our case entity. We also wanted to organize it based on Case Number.

For this, we used KingswaySoft’s SSIS components.

Other posts on SSIS and Dynamics 365 –

https://nishantrana.me/2018/11/26/ssis-and-microsoft-dynamics-365/

This is how our final package looked like

For CDS Source, we used Source Type as Fetch XML.

And the below Fetch XML to fetch notes details where objecttypecode is incident and isdocument property are true (i.e. has an attachment) and case as the linked entity to get the case ticket number.

Result View –

CDS / CRM Source Component

Make sure to use the alias for the attributes of the linked entity.

Next drag the Premium Derived Column, here we will use the DecodeBase64 encoding function to decode the file content stored in the documentbody field using as base-64 encoded string.

DT_Image output column will have the binary content of the attachment

Next, we have added another Premium Derived Column, this time to write the binary content to the local drive, using the WriteBinaryContent function.

Click on the ellipsis to specify the expression.

Here we are creating a folder based on the case ticket number and storing the corresponding attachments inside it.

Let us run the package

On successful execution, we can see the folders created with case ticket number and attachments added to it.

Get all the details here

http://www.kingswaysoft.com/blog/2017/03/21/Extracting-CRM-Attachments-with-Ease

Hope it helps..

An error occurred when processing the security tokens in the message:You are using Ws-Tust authentication which has been deprecated and no longer supported in your environment. Please use oAuth2.0 authentication in Dynamics 365 / CDS


While trying to connect to Dynamics 365 instance through Online Federation authentication type (deprecated) using CDS/ CRM connection manager of KingswaySoft

For Console App – https://nishantrana.me/2020/11/09/sample-code-to-connect-to-cds-dynamics-365-ce-using-oauth/

you would get the below error –

KingswaySoft.IntegrationToolkit.DynamicsCrm.CrmServiceException: CRM service call returned an error: An error occurred when processing the security tokens in the message:You are using Ws-Tust authentication which has been deprecated and no longer supported in your environment. Please use oAuth2.0 authentication. (SSIS Integration Toolkit for Microsoft Dynamics 365, v20.2.0.3083 – devenv, v15.9.28307.905)System.ServiceModel.FaultException

: An error occurred when processing the security tokens in the message:You are using Ws-Tust authentication which has been deprecated and no longer supported in your environment. Please use oAuth2.0 authentication.

The solution is to use the OAuth connection type as shown below.

More on OAuth 2.0 – https://nishantrana.me/2019/08/30/oauth-2-0-with-dynamics-365-ce-web-api/

Here we have selected Password as OAuth type.

And specified username and password, and for Client App Id we have specified the default client app id.

2ad88395-b77d-4561-9441-d0e40824f9bc

https://nishantrana.me/2019/08/23/connect-to-dynamics-365-web-api-using-oauth-2-0-resource-owner-password-credential-ropc/

This connection succeeds this time.

The same will be the experience with XrmToolBox while trying to create a new connection using the connection wizard

Microsoft.Xrm.Tooling.Connector.CrmServiceClient    Error    2    06/11/20 07:18:46 Unable to connect to CRM: An error occurred when processing the security tokens in the message:You are using Ws-Tust authentication which has been deprecated and no longer supported in your environment. Please use oAuth2.0 authentication.

Error    : An error occurred when processing the security tokens in the message:You are using Ws-Tust authentication which has been deprecated and no longer supported in your environment. Please use oAuth2.0 authentication.

Try the alternate connection method like SDK Login Control, Client ID secret, etc.

https://www.xrmtoolbox.com/documentation/for-users/connecting-to-an-organization/

Please refer to the below articles to get more details –

Use of Microsoft 365 authentication with the WS-Trust security protocol

Deprecation of Office365 authentication type and OrganizationServiceProxy class for connecting to Common Data Service

https://colinvermander.wordpress.com/2020/02/09/cds-crm-sdk-ws-trust-auth-and-organizationserviceproxy-deprecated-what-does-it-mean/

Hope it helps..

Advertisements

Using KingswaySoft Azure Blob Storage component with Dynamics 365


KingswaySoft provides 3 components specific to Azure Blog Storage

  • Azure Blob Connection Manager
  • Azure Blob Storage Source
  • Azure Blob Storage Destination

Check other blog posts on SSIS

https://nishantrana.me/2020/10/16/ssis-kingswaysoft-and-dynamics-365/

Let us take a simple example where we are extracting \ reading contact data from Dynamics 365 \ CDS and saving it as a text file in Azure Blob Storage.

Create a new SSIS Package, and drag the Dynamics CRM Source component to it.

(Define the connection to Dynamics CRM instance using Dynamics CRM Connection Manager)

We have selected emailaddress1, first name, and the last name attributes.

Next drag an Azure Blob Destination Component, we will first define the connection to Azure Storage first

Navigate to the Access keys in the existing Storage account to get the account name and the key (or create a new Storage account)

Paste the values and test the connection.

Below we have specified the name of the existing Blob Container, the name of the file to be created there along with the format.

The mapping column will have the mappings defined

Let us execute the package

We can see the file created in the Blob storage

We can see it populated with the source columns specified in CRM Source component.

Similarly using the Azure Blog Source component, we can read the files and can use it for any destination component including Dynamics 365 Destination.

Hope it helps..

Deploy and run SSIS Integration Toolkit for Dynamics 365 on Azure Data Factory (KingswaySoft)


In the previous post, we saw how to deploy and run SSIS packages on the cloud.

Here we take it one step further and will deploy and run the SSIS packages that use KingswaySoft’ s SSIS Integration Toolkit components.

Here we will need an Azure Subscription, where we will host the SSISDB, followed by provisioning Azure-SSIS Integration runtime instance.

We will also need the Azure Blob Storage account along with Azure Storage Explorer to upload the installation files of the SSIS Integration Toolkit.

Let us first start by creating an Azure SQL Server instance.

We have specified the below details.

Now next create the database inside the server.

Now with Azure SQL Server and Database created, the next step is to create the Storage account.

With the Azure Storage created, now let us connect to Azure using the Azure Storage Explorer.

Create a new blob container in the Azure Storage account created.

For the blob container created, right-click and select Get Shared Access Signature

Specify the expiry time along with Write permissions, this is for logging purpose when the Azure-SSIS IR is being provisioned.

Copy the URL (it will be used in the PowerShell script later)

Now let us get the installation files and programs from the KingswaySoft Shared Blob Container, which we’d place in the blob container we just created.

https://kingswaysoftgeneral.blob.core.windows.net/ssis-integration-toolkit-ultimate?st=2019-07-04T16%3A10%3A25Z&se=2059-07-05T16%3A10%3A00Z&sp=rl&sv=2018-03-28&sr=c&sig=LAGvouFpkZHEk%2BH8%2B0pK%2FDNg7B3jPUf%2FJ91%2BJ%2FEeKg0%3D

Right-click Storage Accounts and select Connect to Azure Storage

Select Use a shared access signature (SAS) URI

Paste the KingswaySoft blob container URL.

We can see the below contents added to the blob container.

Select all and copy all the files.

Paste it in the blob container we had created earlier.

With things now setup, let us get the PowerShell script to provision the Azure-SSIS Integration Runtime Initializations.ps1 and update it.

Specify the appropriate values and run the script. Get the Azure PowerShell.

Also, make sure to update the firewall rules to allow the client to connect.

Update the PowerShell Script appropriately

We can check the status as shown below.

In parallel, we can see our Azure Data Factory created with the integration runtime, which is in Starting status.

After a few minutes, we will have integration runtime up and running.

Below is our SSIS Package that we would be deploying to the cloud.

It uses Data Spawner Component to generate test data for Contacts and the CDS Destination component to create those records inside CDS.

Right-click the integration project and select Deploy

Specify connection details along with Path

After successful deployment, let us create a new pipeline inside the Azure Data Factory.

Drag and drop the Execute SSIS Package and click on the Settings tab.

Connect to the package deployed followed by Validate and Debug to test the pipeline.

The pipeline will be in Queued status

After successful execution,

navigate to our Dynamics 365 Sales Hub

We can see 10 contact records created by the SSIS Package.

Hope it helps..

Advertisements