How to set up an AWS Glue ETL Job?

  • Post author:
  • Post category:Azure
  • Post last modified:August 11, 2023
  • Reading time:7 mins read

Let's take a quick end-to-end look at how to convert data from one format to another using AWS Glue, then query that data using AWS Athena. You can download the file used in this tutorial from Kaggle's dataset repository (https://www.kaggle.com/datasets/swathiunnikrishnan/amazon-consumer-behaviour-dataset). Step 1: Start by uploading…

Continue ReadingHow to set up an AWS Glue ETL Job?

Building Efficient Data Pipelines with Azure Data Factory (ADF)

  • Post author:
  • Post category:Azure
  • Post last modified:August 8, 2023
  • Reading time:5 mins read

Introduction In today's data-driven world, organizations rely on seamless and efficient data movement and processing to make informed decisions. Azure Data Factory (ADF) is a powerful cloud-based service from Microsoft that enables data engineers and analysts to create, schedule, and manage data pipelines for orchestrating…

Continue ReadingBuilding Efficient Data Pipelines with Azure Data Factory (ADF)

Copying Multiple Files in a SharePoint Online Folder to Azure data lake using Azure Data Factory

  • Post author:
  • Post category:Azure
  • Post last modified:August 8, 2023
  • Reading time:12 mins read

To copy file from SharePoint Online requires Azure Active Directory/ Service Principal authentication and SharePoint API to retrieve files. Step 1 - Register SharePoint Application and grant permissions. A) Register Azure Active Directory Application 1)  On the Azure Portal go to Azure Active Directory App…

Continue ReadingCopying Multiple Files in a SharePoint Online Folder to Azure data lake using Azure Data Factory

How to automate your organization’s Azure Dev-Ops Power-BI reports?

  • Post author:
  • Post category:AzureGeneral
  • Post last modified:July 13, 2023
  • Reading time:13 mins read

Automating Your Organization’s Azure Dev-Ops Power-BI Reports. Azure Dev-Ops provide us a way of managing, organizing, and tracking all the tasks and their subtasks for a project. Since there is a direct integration between Azure Dev-Ops and Power BI, Power BI can be used to…

Continue ReadingHow to automate your organization’s Azure Dev-Ops Power-BI reports?

Azure Key Vault and its integration into ADF

  • Post author:
  • Post category:Azure
  • Post last modified:July 13, 2023
  • Reading time:6 mins read

Azure key vault is service offered by Microsoft azure to securely store credentials, key, API ,password etc with encryption. Credentials and key stored under azure key vault can be used for authentication across multiple azure service. Key Vault service supports two types of containers: vaults…

Continue ReadingAzure Key Vault and its integration into ADF

End of content

No more pages to load