Cannot convert between unicode and non-unicode string data types error in SSIS

While using the OLE DB Source component in SSIS, on updating its SQL Command we started getting the below error for “Phone” column in our Table.

The fix is to open the Advanced Editor for the OLE DB Source component

Change the Data Type (non-unicode string) for the Phone column inside Output Columns in Input and Output Properties Tab.

Changing the Data Type from Unicode string [DT_WSTR] to string [DT_STR] fixed the issue.

Hope it helps..

How to – convert a blank string to Null for DateTime field – SSIS

Recently while writing an SSIS package, we were getting the below exception.

System.Exception: Error: Failed to set field ‘birthdate’ value to ” for entity ‘contact’ (field type: DateTime, value type: String). System.FormatException: String was not recognized as a valid DateTime. (SSIS Integration Toolkit for Microsoft Dynamics 365, v10.2.0.6972 – DtsDebugHost, v14.0.3025.39)

We were getting the exception because the source table was having the birth date field as a string with blank value instead of null and in the destination table we had the corresponding mapped field as DateTime type.

The fix is to add a Derived Column with the following expression, which converts the blank string to null à

“TRIM(ColumnName) == “” ? (DT_STR,4,1252)NULL(DT_STR,4,1252) : Columnname”


Hope it helps..

The process cannot access the file .ispac because it is being used by another process error in Visual Studio (SSIS).

Recently while developing SSIS packages, the Visual Studio (SSDT) got crashed.

After restarting the Visual Studio and trying to execute the package we got the below error.

System.IO.IOException: The process cannot access the file ‘c:\folder\ssisproject.ispac’ because it is being used by another process.

at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

at System.IO.File.Delete(String path)

at Microsoft.DataTransformationServices.Project.DataTransformationsProjectBuilder. IncrementalBuildThroughObj(IOutputWindow outputWindow)

at Microsoft.DataTransformationServices.Project.DataTransformationsProjectBuilder.

BuildIncremental(IOutputWindow outputWindow)

ISPAC file is the integration services project deployment file containing the packages and the parameters.

The fix is to open task manager and end the DtsDebugHost.exe task.

DtsDebugHost

DtsDebugHost.exe is the SSIS Debug runtime process that executes the package.

Hope it helps..

Using Conditional Split Transformation in SSIS

Recently we had a requirement to Upsert Customer’s Spend related information from SQL DB to our CRM Online.

We had the spend captured per brand wise in the DB as shown below.

And in our CRM, we had one custom entity that had fields corresponding to each of these Brand as shown below

i.e. one section for each Brand and inside that section 5 fields corresponding to the columns in the SQL Table.

The GCR column of the table was the Alternate Key column in our custom entity which we used for Upsert.

Now to make sure we update only the corresponding Brand fields in CRM, we can make use of Conditional Split Transformation component.

Just after OLE DB source, add a Conditional Split component

Create one output per brand i.e. (select distinct Brand from the source table records) as shown below

Now add a CRM Destination Component and bind it to the corresponding output

And inside Columns section, map only the corresponding field of the selected Brand.

The finished package à

Hope it helps..

Optimum Batch Size while using SSIS Integration toolkit for Microsoft Dynamics 365

Hi,

https://nishantrana.me/2018/12/19/optimum-batch-size-and-thread-while-deleting-records-using-ssis-integration-toolkit-for-microsoft-dynamics-365/

Recently while working in a data integration task, we were evaluating the performance of using KingswaySoft’s Dynamics 365 Toolkit from within Visual Studio by trying out different possible combinations of batch size and threads.

The environment was CRM Online and had concurrent ExecuteMultiple max limit set to 20.

By default, it is set as 2 for online, but we can raise support ticket with Microsoft to get this limit increased. However, we need be very careful while using them as large number of ExecuteMultiple running in parallel might degrade the performance.

https://msdn.microsoft.com/en-us/library/microsoft.xrm.sdk.deployment.throttlesettings.executemultipleperorgmaxconnectionsperserver.aspx

Below are the 2 settings that we played around in the CRM Destination Component


These are the results we got with different batch size and thread.

  • Message Used– Upsert.
  • Total Number of Records – 20000.


It clearly shows that having batch size as 100 gives the best result.

And this is what is recommended by KingswaySoft as well.


https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365/faq

 

Hope it helps..

CRM service call returned an error: The server is busy and the request was not completed. Try again later while inserting records using CRM Destination Component of KingswaySoft’s SSIS Integration Toolkit for Dynamics 365

While running one of our package we got the below error in few of the records

“CRM service call returned an error: CRM service call returned an error: The server is busy and the request was not completed. Try again later. (Error Code: -2147180543, Detail Message: The server is busy and the request was not completed. Try again later.)”

“CRM service call returned an error: CRM service call returned an error: The request channel timed out while waiting for a reply after 00:01:59.7991956. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. (Error Type / Reason: Timeout)”

One way to get around this error is to increase the Timeout (secs) in the Service Timeout Setting of CRM Connection Manager.

We updated it to 600 seconds and the number of records that failed got drastically reduced.

Also make sure that following option is checked

Hope it helps..

Lookup not getting set while using CRM Destination Component of KingswaySoft’s SSIS Integration Toolkit for Dynamics 365

Recently while inserting a custom entity (Transaction) records using the CRM Destination Component we realized that Contact’s lookup was not being set.

We were using Alternate Key to set the lookup as shown below

To get it working we had to uncheck the Remove Unresolvable References option from the Optional Settings of CRM Destination Component. It seemed that it ignored it, even though values were there for the GCR (alternate key field) in the input source.

https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365/help-manual/crm/destination

I think it might not be working as expected in case if lookup is done using Alternate Key.

BTW, I would strongly recommend using KingswaySoft’s SSIS Integration Toolkit for any or all data integration tasks. It is definitely the best tool out there.

Give it a try à

https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365/download

Hope it helps..

View Designer not showing SSIS Package in Design mode in Visual Studio

Time to time we’d see our Package not opening in Designer mode in Visual Studio.

To fix this,

Add the package again in the solution or restart the Visual Studio.

Hope it helps..

Using Lookup Transformation and Cache Transformation in SSIS for Dynamics 365 (using Kingswaysoft)

Recently while working in data migration project, we had a requirement to insert a record in CRM from SQL Database and assign it to the team insert.

For e.g.

We had a Transaction Record that had a Brand lookup in it i.e. n – 1 relationship.

And the Brand record had a lookup of Team in it again n – 1. So basically, while inserting the transaction record we wanted to assign the record to the team of the corresponding brand in it.

So, for this, we had to set the owner id and owner id type field while inserting the record.

And for owner id i.e. team GUID we had to do a lookup on the Brand entity.

To achieve so, we did the following step

Added 2 Data flow task in the Control Flow, one for Caching the Brand entity’s detail and other for using the fetching data from SQL DB, Lookup Transformation task and inserting the data into CRM.

For the first data flow task,

We added the Dynamics CRM Source and fetched the Brand Name and Team Lookup’ GUID field.

name field which will be used for mapping and teamid field whose value we need from it.

Then we added a Cache Transform task with following details,

Now for our second data task flow,

First, we added the OLE DB Source to fetch data from the source SQL View, having Brand (string) column in it.

Then we added the Lookup Transformation task with following properties,

Setting Connection to Cache Connection Manager defined earlier.

Now the most important part, the columns properties. Here we defined the mapping between the Brand field of the SQL View and name field of the Brand Entity of CRM and selected the teamid field as the lookup column whose value we would like to fetch as shown below.

Then we added Derived Column task to add one more column for specifying ownerid type field to be used in CRM Destination Component while mapping. We defined the field type as string and hardcoded the value as “team”.

And as the final step mapped the above fields to the corresponding ownerid and owneridtype field in CRM Destination Component for Create operation.

For ownerid, we selected the below option as we were setting the GUID of the team.

On running the package, we can see the transaction records getting created and properly assigned to the corresponding team of the brand.

The transaction record with Team set as Owner.

Brand record with Team lookup in it.

Hope it helps..