eddie · @dreddie
0 followers · 22 posts · Server social.vivaldi.netIt turns out all that was needed in the #dataflow was a custom repartition on all input streams for a join. This took the time down to a mere 34 minutes (from 5 hours). Most likely still fairly slow, but for now it is 'fast enough'.
I still wish to know if there is a way to see statistics or bottlenecks for analyzing this stuff in #datafactory.
eddie · @dreddie
0 followers · 7 posts · Server social.vivaldi.netAny #datafactory experts that can give me some general tips on optimizing performance of a dataflow? I feel that 5 hrs for 30M records is fairly slow on a medium instance.
Nothing too complex happening. Load some data from sql tables, do some simple joins + conversion and send as json to blob storage.
But I can't seem to pinpoint which part specifically is the bottleneck.
SQLAllFather · @SQLAllFather
870 followers · 3372 posts · Server techhub.social
T-minus one week!
Next Thursday I will join the DC Power BI User Group to talk about Microsoft Fabric!
In this hybrid session (I'll be presenting remotely) I'll introduce Fabric and then spend most of the hour walking through an end-to-end solution that showcases new and familiar capabilities. I hope you'll join me!
#PowerBI #MicrosoftFabric #Data #SQL #DataFactory #Spark #lakehouse #warehouse #Free
#powerbi #microsoftfabric #data #sql #datafactory #spark #lakehouse #warehouse #free
2meterdba | Reitse Eskens · @2meterdba
32 followers · 32 posts · Server mastodon.nl
My first impression of the #Azuresql #serverless #hyperscale database. Deployment, loading some data with #datafactory and pricing. #zekerweten
https://sqlreitse.com/2023/02/19/azure-hyperscale-serverless-first-impressions/
#zekerweten #datafactory #Hyperscale #serverless #azuresql
Santosh Hari ( •_•)>⌐■-■ · @santoshhari
307 followers · 1022 posts · Server hachyderm.ioLive shot of #Azure #Synapse and #DataFactory services https://learn.microsoft.com/en-us/azure/synapse-analytics/data-integration/concepts-data-factory-differences
Kevin Chant · @kevchant
213 followers · 177 posts · Server techhub.socialThose who use #Azure #DataFactory are will probably want to read this announcement about the introduction of managed airflow
Kevin Chant · @kevchant
185 followers · 94 posts · Server techhub.social
Published yesterday. Covers where you can find Azure Data Factory objects in Synapse Studio #Azure #DataFactory #AzureSynapse
https://www.kevinrchant.com/2022/12/27/data-factory-objects-in-synapse-studio/
#azure #datafactory #AzureSynapse
Kevin Chant · @kevchant
185 followers · 92 posts · Server techhub.socialCovers where you can find Azure Data Factory objects in Synapse Studio #Azure #DataFactory #AzureSynapse
https://www.kevinrchant.com/2022/12/27/data-factory-objects-in-synapse-studio/
#azure #datafactory #AzureSynapse
Talha Turan · @talhaturan
3 followers · 7 posts · Server mas.toAzure Key Vault nedir? Nasıl kullanılır? https://www.talhaturan.com/azure-key-vault-nedir-nasil-kullanilir/
#azure #bulut #cloud #datafactory #keyvault #secret
#secret #keyvault #datafactory #cloud #bulut #azure
Talha Turan · @talhaturan
3 followers · 11 posts · Server mas.toAzure Key Vault nedir? Nasıl kullanılır? https://www.talhaturan.com/azure-key-vault-nedir-nasil-kullanilir/
#azure #bulut #cloud #datafactory #keyvault #secret
#secret #keyvault #datafactory #cloud #bulut #azure
Talha Turan · @talhaturan
0 followers · 5 posts · Server mas.toAzure Data Factory ile Analysis Services ölçeklendirme
https://www.talhaturan.com/azure-data-factory-ile-analysis-services-olceklendirme/
#azure #bulut #cloud #datafactory #analysisservices #dataengineering
#dataengineering #analysisservices #datafactory #cloud #bulut #azure
Talha Turan · @talhaturan
3 followers · 11 posts · Server mas.toAzure Data Factory ile Analysis Services ölçeklendirme
https://www.talhaturan.com/azure-data-factory-ile-analysis-services-olceklendirme/
#azure #bulut #cloud #datafactory #analysisservices #dataengineering
#dataengineering #analysisservices #datafactory #cloud #bulut #azure
Talha Turan · @talhaturan
3 followers · 11 posts · Server mas.toLogic App ile Azure sanal makinelerini başlatma
https://www.talhaturan.com/logic-app-ile-azure-sanal-makinelerini-baslatma/
#azure #bulut #cloud #datafactory #logicapp #dataengineering #datafactory
#dataengineering #LogicApp #datafactory #cloud #bulut #azure
Talha Turan · @talhaturan
0 followers · 3 posts · Server mas.toAzure Logic App ve Data Factory kullanarak e-posta ile bilgilendirme akışı https://www.talhaturan.com/azure-logic-app-ve-data-factory-kullanarak-e-posta-ile-bilgilendirme-akisi/
#azure #bulut #cloud #datafactory #logicapp #dataengineering
#dataengineering #LogicApp #datafactory #cloud #bulut #azure
Talha Turan · @talhaturan
3 followers · 11 posts · Server mas.toAzure Logic App ve Data Factory kullanarak e-posta ile bilgilendirme akışı https://www.talhaturan.com/azure-logic-app-ve-data-factory-kullanarak-e-posta-ile-bilgilendirme-akisi/
#azure #bulut #cloud #datafactory #logicapp #dataengineering
#dataengineering #LogicApp #datafactory #cloud #bulut #azure