Create dependent pipelines in your Azure Data Factory
Data integration is complex and has many moving parts that spans across hybrid data environments. Typically, data integration projects have dependencies upstream and downstream making dependencies an important aspect to consider in any job scheduling.Gaurav Malhotra joins Scott Hanselman to ... continue reading

Build ETL pipelines collaboratively using Git integration in Azure Data Factory
Enable Git source control (Azure DevOps Git or GitHub) in your data factories to do collaboration, source control, change tracking, change difference, continuous integration, and deployment.Visual authoring in Azure Data Factory - Advantages of Git integrationAzure Data Factory overviewAzure Data ... continue reading
How to see Azure Alerts: Query rules and Activity Log in SCOM through Azure Management Pack
Azure Management pack https://www.microsoft.com/en-us/download/details.aspx?id=50013 is designed to enable hybrid scenario where SCOM is used as a single pane of glass to monitor both Azure and on-prem resources. If you want to monitor your Azure resources like Virtual machines, Storage accounts, ... continue reading

Code-free modern data warehouse using Azure SQL DW and Data Factory
Gaurav Malhotra joins Scott Hanselman to show how to build a modern data warehouse solution from ingress of structured, unstructured, semi-structured data to code-free data transformation at scale and finally to extracting business insights into your Azure SQL Data Warehouse.Azure ... continue reading

Move your data from AWS S3 to Azure Storage using AzCopy
AzCopy v10 (Preview) now supports Amazon Web Services (AWS) S3 as a data source. You can now copy an entire AWS S3 bucket, or even multiple buckets, to Azure Blob Storage using AzCopy. Customers who wanted to migrate their data ... continue reading
High-Throughput with Azure Blob Storage
I am happy to announce that High-Throughput Block Blob (HTBB) is globally enabled in Azure Blob Storage. HTBB provides significantly improved and instantaneous write throughput when ingesting larger block blobs, up to the storage account limits for a single blob ... continue reading

Rerun activities inside your Azure Data Factory pipelines
Data Integration is complex with many moving parts. It helps organizations to combine data and complex business processes in hybrid data environments. Failures are very common in data integration workflows. This can happen due to data not arriving on time, ... continue reading

Quickly build data integration pipelines using templates in Azure Data Factory
Gaurav Malhotra joins Lara Rubbelke (@sqlgal) to show how to use predefined Azure Data Factory pipeline templates to get started quickly with Azure Data Factory thereby reducing the development time for building data integration projects.Overview of templates for Azure Data ... continue reading

PyTorch on Azure: Deep learning in the oil and gas industry
This blog post was co-authored by Jürgen Weichenberger, Chief Data Scientist, Accenture and Mathew Salvaris, Senior Data Scientist, Microsoft Drilling for oil and gas is one of the most dangerous jobs on Earth. Workers are exposed to the risk of ... continue reading

Code-free data transformation at scale using Azure Data Factory
Learn about the new code-free visual data transformation capabilities in Azure Data Factory as Gaurav Malhotra joins Lara Rubbelke to demonstrate how you can visually design, build, and manage data transformation processes without learning Spark or having a deep understanding ... continue reading