Data integration is complex and has many moving parts that spans across hybrid data environments. Typically, data integration projects have dependencies upstream and downstream making dependencies an important aspect to consider in any job scheduling.Gaurav Malhotra joins Scott Hanselman to ... continue reading

Enable Git source control (Azure DevOps Git or GitHub) in your data factories to do collaboration, source control, change tracking, change difference, continuous integration, and deployment.Visual authoring in Azure Data Factory - Advantages of Git integrationAzure Data Factory overviewAzure Data ... continue reading

Azure Management pack https://www.microsoft.com/en-us/download/details.aspx?id=50013 is designed to enable hybrid scenario where SCOM is used as a single pane of glass to monitor both Azure and on-prem resources. If you want to monitor your Azure resources like Virtual machines, Storage accounts, ... continue reading

Gaurav Malhotra joins Scott Hanselman to show how to build a modern data warehouse solution from ingress of structured, unstructured, semi-structured data to code-free data transformation at scale and finally to extracting business insights into your Azure SQL Data Warehouse.Azure ... continue reading

AzCopy v10 (Preview) now supports Amazon Web Services (AWS) S3 as a data source. You can now copy an entire AWS S3 bucket, or even multiple buckets, to Azure Blob Storage using AzCopy. Customers who wanted to migrate their data ... continue reading

I am happy to announce that High-Throughput Block Blob (HTBB) is globally enabled in Azure Blob Storage. HTBB provides significantly improved and instantaneous write throughput when ingesting larger block blobs, up to the storage account limits for a single blob ... continue reading

Data Integration is complex with many moving parts. It helps organizations to combine data and complex business processes in hybrid data environments. Failures are very common in data integration workflows. This can happen due to data not arriving on time, ... continue reading

Gaurav Malhotra joins Lara Rubbelke (@sqlgal) to show how to use predefined Azure Data Factory pipeline templates to get started quickly with Azure Data Factory thereby reducing the development time for building data integration projects.Overview of templates for Azure Data ... continue reading

This blog post was co-authored by Jürgen Weichenberger, Chief Data Scientist, Accenture and Mathew Salvaris, Senior Data Scientist, Microsoft Drilling for oil and gas is one of the most dangerous jobs on Earth. Workers are exposed to the risk of ... continue reading

Learn about the new code-free visual data transformation capabilities in Azure Data Factory as Gaurav Malhotra joins Lara Rubbelke to demonstrate how you can visually design, build, and manage data transformation processes without learning Spark or having a deep understanding ... continue reading