Thanks KingswaySoft

Thanks KingswaySoft for coming up with one of the best tools for Data Integration and especially the SSIS Integration Toolkit for Microsoft Dynamics 365..

https://www.kingswaysoft.com/partners/MVPsMVP_SSIS

Advertisements

System.AggregateException: One or more errors occurred. (SSIS Integration Toolkit for Microsoft Dynamics 365) error in KingswaySoft’s SSIS Integration Toolkit

Occasionally we might get the below error in our SSIS Package.

[Dynamics CRM Destination [2]] Error: An error occurred with the following error message: “System.AggregateException: One or more errors occurred. (SSIS Integration Toolkit for Microsoft Dynamics 365, v10.2.0.6982 – DtsDebugHost, v14.0.3002.113)KingswaySoft.IntegrationToolkit.DynamicsCRM.CrmServiceException

: CRM service call returned an error: CRM service call returned an error: The request channel timed out while waiting for a reply after 00:01:59.6189794. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. (Error Type / Reason: Timeout)KingswaySoft.IntegrationToolkit.DynamicsCRM.CrmServiceException

(Status Reason: Timeout): The operation has timed out”.

The way we can fix is either increase the time out in the CDS/CRM Connection Manager as suggested.

However the better solution is to keep the batch size low around 100 or so in the CDS/CRM Destination Component. We have observed that decreasing the batch size helps here in case of this error.

Hope it helps..

CrmRecordId and IsNew Output in CRM/CDS Destination Component

In CRM/CDS Destination Component of SSIS Integration toolkit we can found 2 output column as part of error handling

In case of Create

  • CrmRecordId will have the GUID of the newly created record, which could be used for any further processing
  • IsNew will list whether this is a new record or not.

Our package: –

Suppose we have the following record in our source (here ID is the alternate key column used for Upsert)

If all the records are new and are created we will get the following output.

Now if we delete the Test User 2 and Test User 3 record and run the package again, which should now create 2 records and update one record i.e. Test User 1 during Upsert.

The output would be as below

It will return the GUID of the record irrespective of Update or Create and IsNew will specify whether the record is newly created or updated.

Hope it helps..

The following job steps cannot be reached with the current job step flow logic in SSIS

Recently while scheduling our SSIS Packages in SQL Server Agent Jobs we got the below error

WARNING: The following job steps cannot be reached with the current job step flow logic:

[1] Fashion Spend package

We just need to make sure that we have set correct package in the Start Step properties in Job Properties Dialog Box

To: –

Hope it helps..

Monitoring SSIS package Job Status

Suppose we have scheduled our packages through SQL Server Agent Job and we receive the notification that one of our packages has failed.

To get its details, within SSISDB, we can check the Standard Reports like Integration Services Dashboard report, All Executions etc. to get the details.

This will provide the information about the operation that have run or are currently executing.

We can click on Failed record link to filter the report.

Or we can Filter it through Filter Settings dialog box as shown below

Open up the report for which we want to check the messages and click on View Messages link to get the details.

We can also view these standard reports (All Execution and All Validations) for a specific project as well.

We can also make use of Active Operation Dialog Box to check the status of currently running SSIS packages and stop it if needed.

Hope it helps..

Solved – Microsoft.SqlServer.Dts.Pipeline.ComponentVersionMismatchException: The version of the Script Component is not compatible with this version of the DataFlow

While trying to run one of our SSIS Packages from SQL Server Job, which had script component in it, we got the below error. It was running fine within the SSDT in our Dev Machine. In fact, the other packages deployed to SSISDB were also running fine, the ones which were not using the Script Component.

To fix it, we updated the project version of SSDT to match the SQL Server where we were deploying the package inside Project Properties.

To find the SQL Server version.

And deployed
only that package instead of the project in SSISDB.

However, on running the package from within the SQL Server again, threw the same error. We tried a few other things like opening the same package in SSDT in the Server and then try deploying that particular package from there. We also tried by deleting the existing the script component in the package and using the script component that was available in the toolbox in SSDT in the Server.

The package clearly showed the difference in the version for the Script Component. However again deploying that single updated package gave the same mismatch exception.

Eventually, we then deployed the Project (after updating the Target SQL Server version to the Server’s SQL version) and not the individual package. And it ran successfully this time.

So basically we need to make sure our target version is correct and deploy the entire Project to fix this issue in our case.

Hope it helps..

Deploying Package to SQL Server Integration Services Catalog (SSISDB) from Visual Studio (SSDT)

Deploying packages to SQL Server from SSDT is straightforward. We can either deploy the project or an individual SSIS Package i.e. Project Deployment or Package Deployment. (SQL Server 2016 Onwards).

Here we will see the package deployment.

Right-click the package that we would like to deploy and select Deploy Package.

This opens the Integration Services Deployment Wizard

Click on Next and Specify Server Name and the credentials to connect to the SQL Server an click on Connect.

For the Path specify the existing Project or create a new Project in SSID where we would like to deploy our packages.

Review the details and if all the information is correct, click on Deploy.

The result page would show the status of the deployment.

Inside Integration Service Catalogs we can see our Package Deployed.

Another option for deploying the package is through the Deploy Project option of the Project inside SQL Server Management Studio’s SSIDB node.

It will open the Integration Service Deployment Wizard.

And after selecting the Package folder we can click on Next and follow the wizard as we had done earlier to deploy the package.

We can also run the Wizard directly from Windows Explorer or command prompt – IsDeploymentWizard.exe

Through Stored Procedure à

catalog.deploy_packages

[catalog].[deploy_packages] [ @folder_name = ] folder_name, [ @project_name = ] project_name, [ @packages_table = ] packages_table, [ @operation_id OUTPUT ] operation_id OUTPUT ]

Through Management Object Model API.

Hope it helps..

Audit Information not getting retrieved using SSIS/KingswaySoft Adapter

Update – 29 Oct 2018 –-> We are getting the same issue in one of the entities and it has audit enabled. We have a raised a support ticket for it. Will update the post with the solution.

A few days back I wrote about how we can use CDS/CRM Source component of KingswaysSoft Adapter to get the audit information.

https://nishantrana.me/2018/10/08/using-kingswaysofts-cds-crm-source-component-to-get-audit-information-in-dynamics-365-ce-ssis/

Recently while writing a package for getting audit details against one of the entities, we realized that the records were not getting retrieved and also, we were not getting any error or exception. The package kept on running.

Setting timeout in the CRM Connection Manager also didn’t stop the package, it kept running without retrieving the records.

After spending a good amount of time, we realized the silly mistake that we had made. We were running the package against the entity for which Audit was not enabled.

Thanks to KingswaySoft Support team for their relentless support.

Ideally, we should have been more careful before running the package against that entity, and also it will be helpful if the tool can also check if the audit is enabled for an entity before beggning the execution and if the audit is not enabled inform back the user or at least return 0 rows and complete its execution successfully.

Hope it helps..

The collection of variables locked for read and write access is not available outside of PostExecute error in SSIS

We might get this error while trying to access a ReadWrite variable within Script Component outside Post Execute.

Here we were using it within ProcessInputRow method

As per the official documentation – the reason for this error is because ReadWriteVariables are only available inside PostExecute method for improved performance and minimizing locking.

https://docs.microsoft.com/en-us/sql/integration-services/extending-packages-scripting/data-flow-script-component/using-variables-in-the-script-component?view=sql-server-2017

Updating the code to use PostExecute method fixed the issue for us.

Hope it helps..

Using OverriddenCreatedOn or Record Created On field to update Created On field in Dynamics 365

While working in data migration project, when creating records in the target system, we would want the “created on” field to hold the original value instead of it being set to the actual value when it was created in the target system, which is set by the system or the platform itself.

Suppose we are having the “created on” field in our source SQL Server Table and we are creating the lead records in our target Dynamics 365 organization using KingswaySoft’s CRM Destination Component.

Here we have mapped the CreatedDate of our source table to overrriddecnreatedon field of Lead entity.

On executing the package, back inside Dynamics 365, we can see the following values for Created On and Record Created On (overrriddecnreatedon) field for the lead records.

Record created on will have the time when the record was created in Dynamics 365 and Created On field will have the values that we passed from our source SQL Table.

Get all the details here

https://blogs.msdn.microsoft.com/emeadcrmsupport/2012/08/01/the-truth-about-override-created-on-or-created-by-for-records-during-data-import/

Hope it helps..