Recently we configured Data Export Service in our Production environment. The idea was to export the data to Azure SQL DB that can be used for Power BI report.
- Easy to configure.
- Takes care of incremental changes.
- Easy to check the status
Issues with the Failed records.
We have around 200K records failed for one entity as shown below.
There is preview feature added for retrying the sync for the failed records.
Clicking on Resync Failed Records didn’t help much here as we didn’t see any change in the count of failed records.
And now we are getting the below message for it.
It has been more than 24 HOURS when we had started the Failed Records Synchronization.
This feature being still in preview, we can expect few things not working.
So how do we deal with failed records, how do we figure out the reason for them failing so that we could correct it and retry our export ?
For this we can use the Failed Records feature of Data Export Service to get the logs of the failed records
It will give us a Blob URL, which is valid for 24 hours, to which we can connect through Azure Storage Explorer and check the logs.
Copy the Blob URL
Download the Azure Storage Explorer
Open Azure Storage Explorer, select the Account section and click on Add an Account.
Select “Use a shared access signature URI” option and click on Next.
Paste the copied Blob URL
Click Next to get the connection summary and click Connect.
After connecting we should be able to see the log
However in our scenario, the blog container was always blank. Ideally it should have the following information
One more limitation that we need to be aware of is that Activity type entities are not supported.
And also if we delete one of the entities from the export profile and plan to add it later to the same profile or to a new profile, we need to delete its table and corresponding user defined type from the Azure SQL DB else we will get the exception.
Considering how easy it is to configure and takes care of the incremental data, this feature is well worth considering, however it is the failed records things that could prove a challenge, if we don’t have the logs for them or we are not able to fix them as we do not have much control over the things when compared to writing a custom code or using some tool like KingswaySoft Adapter.
Hope this helps..