Transfer files from local drive to Azure Blob using Premium File Transfer Task – SSIS

The Premium File Transfer Task component of KingswaySoft can be used to easily transfer files from local drive to Azure Blob storage.

Let us take a simple example to see it in action.

Here we will pick the folder Customers and its subfolders along with the files inside it and will transfer it’s content to the Azure Blob Container.

Create a new SSIS Package and drag the Premium File Transfer Task to the control flow designer.

Double click the task and specify the following Source Properties

  • Action – Send Files – the other options are – Delete files, Create Directory, Remove Directory.
  • Check the option – Include Subdirectories
  • Connection Manager – Local File
  • Directory Path – specify the folder

Similarly, for Destination, we can specify the Azure Blog Storage Connection Manager and the directory path as shown below

Note – Make sure we have already added the connection for it to be available inside the connection manager option of Premium File Transfer Task. The other connection types supported are FTPS, SFT, Amazon S3, Azure Data Lake Storage, Box, Dropbox, Hadoop, OneDrive, SharePoint.

Let us run the package.

We can see the content transferred successfully to Azure Blog storage



Also, check out –

Using Azure Blob Storage component with Dynamics 365

Hope it helps..

Process Optimization – Dynamics 365 CE / CRM / CDS SSIS Integration toolkit – KingswaySoft

Process Optimizations is a new feature added in the CDS / CRM Destination Component of KingswaySoft’s SSIS Integration Toolkit as part of November 2020 Release.

Check other posts on SSIS and Dynamics 365

These options if enabled, will turn off or disable plugin, workflow, entity auditing during pre-execution temporarily, during the writing process, and will revert the changes to the original state after successful execution.

  • Tentatively Disable Relevant Plugins (if any)
  • Tentatively Disable Relevant Workflows (if any)
  • Tentatively Disable Relevant Auditing (if any)

Let us take a simple example to see it in action.

Here we have a simple data flow that creates contact records inside CDS using Data Spawner.

Let us run the package and see the status of the plugin, workflow, and audit with regards to the contact entity.

As expected, we do not see any Audit records created for contact records creation.

We see an event where Audit is disabled and then enabled later for the contact entity, during the duration of package running and after its successful completion.

We can also see our workflow in status – Draft

In the case of Plugin, we didn’t see the step being disabled and it was still being triggered

As per the documentation –

Get the toolkit here

Hope it helps..

Changes in CDS / CRM Destination Component – SSIS Integration Toolkit for Dynamics 365

With version 20.2 November 2020 release, there have few updates added to the KingwaySoft’s CDS/CRM Destination Component, few are changes in the label and others are metadata changes, that one should be aware of before updating.

Check Post on SSIS and Dynamics 365

  • Label changes.

Version 20.1 –

Version 20.2 –

  • CrmRecordId renamed to SavedRecordId

Version 20.1 –

Version 20.2 –

  • CrmErrorMessage renamed to ErrorMessage

Version 20.1 –

Version 20.2 –

Thus, any package saved with the new version will not work with the older version, and it is recommended to take full back up of the package before updating to the new version.

Hope it helps..

How to – Extract attachment from Notes in Dynamics 365 / CE / CDS – SSIS (KingswaySoft)

Recently we were exploring ways of extracting attachment from notes to a local drive for our case entity. We also wanted to organize it based on Case Number.

For this, we used KingswaySoft’s SSIS components.

Other posts on SSIS and Dynamics 365 –

This is how our final package looked like

For CDS Source, we used Source Type as Fetch XML.

And the below Fetch XML to fetch notes details where objecttypecode is incident and isdocument property are true (i.e. has an attachment) and case as the linked entity to get the case ticket number.

Result View –

CDS / CRM Source Component

Make sure to use the alias for the attributes of the linked entity.

Next drag the Premium Derived Column, here we will use the DecodeBase64 encoding function to decode the file content stored in the documentbody field using as base-64 encoded string.

DT_Image output column will have the binary content of the attachment

Next, we have added another Premium Derived Column, this time to write the binary content to the local drive, using the WriteBinaryContent function.

Click on the ellipsis to specify the expression.

Here we are creating a folder based on the case ticket number and storing the corresponding attachments inside it.

Let us run the package

On successful execution, we can see the folders created with case ticket number and attachments added to it.

Get all the details here

Hope it helps..


Replicate / Export Dynamics 365 (CDS) data – different options

The most common use case to export Dynamics 365 or Common Data Service data to an external database/datastore is to have more control over the data (without going through API route) with the ability to run analytics such as Power BI Reporting, Machine Learning, Data Warehousing, Integration, etc.

For analytics, we can also use our Microsoft Dynamics 365 AI apps that use Dynamics 365 data as a source

Below listing down few of the options –

Data Export Service – Configuration

It is a free Add on-service provided by Microsoft for exporting Dynamics 365 or CDS Data to Microsoft Azure SQL Database or Microsoft Azure SQL Server on Microsoft Azure Virtual Machine.

Microsoft recommends Azure SQL Database Premium P1 or a better plan for Data Export Service

Notes on Data Export Service – 

Export to Data Lake Service – Configuration

The service enables continuous replication of CDS Data to Azure Data Lake Gen 2.

Few posts on Azure Data Lake

Azure Data Factory – GUI based integration tool

Cloud-based integration service.

Check out the informative posts on Azure Data Factory –

We can also run SSIS Packages on Azure Data Factory

Scribe Online Replication Services – GUI based integration tool

Cloud-based integration service.

Another option is to use Scribe Insight

Skyvia’s Data Integration service


Cloud-based integration service.

Another GUI based integration tool that can be considered is Informatica

KingswaySoft Integration Components – SSIS

Leading data integration software – provide SSIS components to integrate applications/databases.

Another SSIS based option is CozyRoc

Or we can use Script Component instead of using 3rd party components/ tool

More on SSIS and Dynamics 365

Use SQL to query CDS data (Preview)

The capability to query CDS Data.

Check out all the articles on CDS T-SQL

Query CDS data using Web API / Organization Service – Development

Write a custom tool using Dynamics 365 APIs

Hope it helps..


Using KingswaySoft Azure Blob Storage component with Dynamics 365

KingswaySoft provides 3 components specific to Azure Blog Storage

  • Azure Blob Connection Manager
  • Azure Blob Storage Source
  • Azure Blob Storage Destination

Check other blog posts on SSIS

Let us take a simple example where we are extracting \ reading contact data from Dynamics 365 \ CDS and saving it as a text file in Azure Blob Storage.

Create a new SSIS Package, and drag the Dynamics CRM Source component to it.

(Define the connection to Dynamics CRM instance using Dynamics CRM Connection Manager)

We have selected emailaddress1, first name, and the last name attributes.

Next drag an Azure Blob Destination Component, we will first define the connection to Azure Storage first

Navigate to the Access keys in the existing Storage account to get the account name and the key (or create a new Storage account)

Paste the values and test the connection.

Below we have specified the name of the existing Blob Container, the name of the file to be created there along with the format.

The mapping column will have the mappings defined

Let us execute the package

We can see the file created in the Blob storage

We can see it populated with the source columns specified in CRM Source component.

Similarly using the Azure Blog Source component, we can read the files and can use it for any destination component including Dynamics 365 Destination.

Hope it helps..

%d bloggers like this: