Select Page
Monitor pipeline

Rerun activities inside your Azure Data Factory pipelines

Data Integration is complex with many moving parts. It helps organizations to combine data and complex business processes in hybrid data environments. Failures are very common in data integration workflows. This can happen due to data not arriving on time, ... continue reading
Quickly build data integration pipelines using templates in Azure Data Factory

Quickly build data integration pipelines using templates in Azure Data Factory

Gaurav Malhotra joins Lara Rubbelke (@sqlgal) to show how to use predefined Azure Data Factory pipeline templates to get started quickly with Azure Data Factory thereby reducing the development time for building data integration projects.Overview of templates for Azure Data ... continue reading
image

PyTorch on Azure: Deep learning in the oil and gas industry

This blog post was co-authored by Jürgen Weichenberger, Chief Data Scientist, Accenture and Mathew Salvaris, Senior Data Scientist, Microsoft Drilling for oil and gas is one of the most dangerous jobs on Earth. Workers are exposed to the risk of ... continue reading
Code-free data transformation at scale using Azure Data Factory

Code-free data transformation at scale using Azure Data Factory

Learn about the new code-free visual data transformation capabilities in Azure Data Factory as Gaurav Malhotra joins Lara Rubbelke to demonstrate how you can visually design, build, and manage data transformation processes without learning Spark or having a deep understanding ... continue reading
Enhanced monitoring capabilities and tags/annotations in Azure Data Factory

Enhanced monitoring capabilities and tags/annotations in Azure Data Factory

Gaurav Malhotra and Scott Hanselman (@shanselman) explore tagging support and enhanced monitoring capabilities, including dashboards and improved debugging support in Azure Data Factory. Data integration is complex and the ability to monitor your data factory pipelines is a key requirement ... continue reading
Monitor your Azure Data Factory pipelines proactively with alerts

Monitor your Azure Data Factory pipelines proactively with alerts

Data Integration is complex and helps organizations to combine data and complex business processes in hybrid data environments. The increase in volume, variety and velocity of data has led to delays in monitoring and reacting to issues. Organizations want to ... continue reading
Run Azure Functions from Azure Data Factory pipelines

Run Azure Functions from Azure Data Factory pipelines

Azure Functions is a serverless compute service that enables you to run code on-demand without having to explicitly provision or manage infrastructure. Using Azure Functions, you can run a script or piece of code in response to a variety of ... continue reading
Creating new alert rule in Azure Data Factory

Create alerts to proactively monitor your data factory pipelines

Data integration is complex and helps organizations combine data and business processes in hybrid data environments. The increase in volume, variety, and velocity of data has led to delays in monitoring and reacting to issues. Organizations want to reduce the ... continue reading
2018-12-03_12h14_00

Azure Functions now supported as a step in Azure Data Factory pipelines

Azure Functions is a serverless compute service that enables you to run code on-demand without having to explicitly provision or manage infrastructure. Using Azure Functions, you can run a script or piece of code in response to a variety of ... continue reading
Cold path processing components of the solution guide

Extracting insights from IoT data using the cold path data flow

This blog continues our coverage of the solution guide published by Microsoft’s Industry Experiences team. The guide covers the following components: Ingesting data Hot path processing Cold path processing Analytics clients We already covered the recommendation for processing data for ... continue reading