Azure Data Factory For Data Engineers – Project on Covid19

Nu kopen

Prijs: $34.99

Welkom!

I am looking forward to helping you with learning one of the in-demand data engineering tools in the cloud, Azure-gegevensfabriek (ADF)! This course has been taught with implementing a data engineering solution using Azure Data Factory (ADF) for a real world problem of reporting Covid-19 trends and prediction of the spread of this virus.

This is like no other course in Udemy for Azure Data Factory or Data Engineering Technologies. Once you have completed the course including all the assignments, I strongly believe that you will be in a position to start a real world data engineering project on your own and also proficient on Azure Data Factory (ADF).

I have also included lessons on the storage solutions such as Azure Data Lake Storage, Ik voeg constant nieuwe inhoud toe in termen van zowel lezingen als demo's, Azure SQL Database etc. Ook, there are lessons on Azure HDInsight and Azure Databricks. I have even included lessons on building reports using Power BI on the data processed by the Azure Data Factory data pipelines. I have considered the machine learning models to be out of scope. You can use this data to build your own models and predict the spread.

The course follows a logical progression of real world project implementation with technical concepts being explained and the data pipelines in Azure Data Factory (ADF) being built at the same time. Even-though this course is not specifically designed to teach you the skills required for passing the Azure Data Engineer Associate Certification exams DP200 & DP203, it can greatly help you get most of the necessary skills required for the exam.

I value your time as much as I do mine. Zo, I have designed this course to be fast-paced and to the point. Ook, the course has been taught with simple English and no jargons. I start the course from basis and by the end of the course you will be proficient in the technologies used.

Currently the course teaches you the following

Azure-gegevensfabriek

• Building a solution architecture for a data engineering solution using Azure Data Engineering technologies such as Azure Data Factory (ADF), Azure Data Lake Gen2, Ik voeg constant nieuwe inhoud toe in termen van zowel lezingen als demo's, Azure SQL-database, Azure Databricks, Azure HDInsight and Microsoft PowerBI.

• Integrating data from HTTP clients, Azure Blob Storage and Azure Data Lake Gen2 using Azure Data Factory.

• Branching and Chaining activities in Azure Data Factory (ADF) Pipelines using control flow activities such as Get Metadata. Als voorwaarde, ForEach, Verwijderen, Validation etc.

• Parameters en variabelen gebruiken in pijplijnen, Gegevenssets en LinkedServices om metagegevensgestuurde pijplijnen te creëren in Azure Data Factory (ADF)

• Debuggen van de datapijplijnen en het oplossen van problemen.

• Pijplijnen plannen met behulp van triggers zoals Event Trigger, Schematrigger en tumblingvenstertrigger in Azure Data Factory (ADF)

• Mappinggegevensstromen creëren om transformatielogica te creëren. De cursus behandelt alle transformatiestappen, zoals Source, Filter, Selecteer, Scharnier, Opzoeken, Voorwaardelijke splitsing, Afgeleide kolom, Totaal, Join and Sink-transformatie.

• Foutopsporing in gegevensstromen, het onderzoeken van problemen, het oplossen van storingen enz

• Implementatie van Azure Data Factory-pijplijnen om toewijzingsgegevensstromen aan te roepen en uit te voeren.

• ADF-pijplijnen maken om HDInsight-activiteiten uit te voeren en gegevenstransformaties uit te voeren.

• Creating ADF pipelines to execute Databricks Notebook activities to carry out transformations.

• Creating dependency between pipelines to orchestrate the data flow

• Creating dependency between triggers to orchestrate the data flow

• Monitoring data pipelines, creating alerts, reporting of metrics from the Azure Data Factory Monitor.

• Monitoring of Data Factory pipelines using Azure Monitor and setting diagnostic setting to be forwarded to Azure Storage Account or Log Analytics Workspace.

• Creating Log Analytics workspace, creating workbooks and charts from log analytics on the Azure Data Factory pipelines

• Implementing the Azure Data Factory Analytics monitoring tool and how to extend the capability further.

Azure Storage Solutions

• Creating Azure Storage Account, Creating containers, Uploading data, Access Control (IAM), Using Azure Storage explorer to interact with the storage account

• Creating Azure Data Lake Gen2, Creating containers, Uploading data, Access Control (IAM), Using Azure Storage explorer to interact with the storage account

• Creating Azure SQL Database, Pricing Tiers, Creating Admin User, Creating Tables, Loading Data and Querying the database.

Azure HDInsight & Deze module bereidt je voor op het leren van Big Data in Azure Cloud met behulp van HDInsight

• Creating HDInsight Clusters, Interacting with the UI, Using Ambari, Creating Hive tables, Invoking HDInsight activities from Azure Data Factory

• Creating Azure Databricks Workspace, Creating Databricks clusters, Mounting storage accounts, Creating Databricks notebooks, performing transformations using Databricks notebooks, Invoking Databricks notebooks from Azure Data Factory.