Recently we configured the Export to Data Lake service for one of our projects.
Just sharing the performance, we got during the initial sync.
|
Entity |
Count |
|
Contact |
2,36,2581 |
|
Custom entity |
1,61,3554 |
The sync started at 11:47 A.M and was completed around 4:50 P.M. – around 5 hours i.e. 300 minutes
Let us consider total records synced as 3975000 – 1613446 + 2362581 + 2 = 3976029.
So the performance here comes down to
- 795000 records per hour
- 13250 records per minutes
- 220 records per second.
Of course, it will vary depending on the specific environment, table / entity type, attributes in it etc. it gives us a rough idea.
Hope it helps..
Advertisements
Nishant Rana's Weblog 
Hi Nishant,
How does the addition or new fields work in the Export to Data Lake (CSV files) ?.
LikeLiked by 1 person
When you add a new column to a table in the source, it is also added at the end of the file in the destination in the corresponding file partition. While the rows that existed prior to the addition of the column won’t show the new column, new or updated rows will show the newly added column.
LikeLike
https://docs.microsoft.com/en-us/powerapps/maker/data-platform/export-data-lake-faq#what-happens-when-i-add-a-column
LikeLiked by 1 person
Thanks a lot Nishant !!
LikeLike
Hi Nishant,
Have you any experience with the “Append Only” export? What column(s) do you query to get the “current” record when reporting?
Regards,
Ben
LikeLike